2023-10-16T08:24:19.798 INFO:root:teuthology version: 0.0.1.dev197+g8cdab07 2023-10-16T08:24:19.799 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/mchangir-2023-10-16_08:10:12-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428951 --name mchangir-2023-10-16_08:10:12-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi --owner mchangir --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} -- /home/teuthworker/archive/mchangir-2023-10-16_08:10:12-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428951/orig.config.yaml 2023-10-16T08:24:19.845 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T08:24:19.924 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/mchangir-2023-10-16_08:10:12-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428951 branch: wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: mchangir@redhat.com first_in_suite: false job_id: '7428951' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: mchangir-2023-10-16_08:10:12-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: c7471068426370a3754c2250e7729280233d552b ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: c7471068426370a3754c2250e7729280233d552b thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: mgr-snap_schedule-support-subvol-and-group-arguments sha1: 6d9e81bfe885e1cd43cce3f41c9616ed0dce2cf3 owner: mchangir priority: 25 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 748 sha1: c7471068426370a3754c2250e7729280233d552b sleep_before_teardown: 0 suite: fs suite_branch: mgr-snap_schedule-support-subvol-and-group-arguments suite_path: /home/teuthworker/src/github.com_mchangir_ceph_6d9e81bfe885e1cd43cce3f41c9616ed0dce2cf3/qa suite_relpath: qa suite_repo: https://github.com/mchangir/ceph.git suite_sha1: 6d9e81bfe885e1cd43cce3f41c9616ed0dce2cf3 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi040.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC9NQL/MGgwiIOjSeKTp24cQ97ZwOsJMk9LstleBGMYYh52MDJnTBeLIx4/B8MBdtdmz/qKA4NUnTfa70peh9nk= smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIPx1U1nPl6AeNRotZqILZpt6k01E0TmhKhCzRMRxWKyrwm7FoM4l+t0BUkfIsIT2DsTx0DwhEW/OSe+eW4DzRo= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 8cdab074dcca9a68965bc5a50e9c30b691949723 timestamp: 2023-10-16_08:10:12 tube: smithi user: mchangir verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-16T08:24:19.924 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_mchangir_ceph_6d9e81bfe885e1cd43cce3f41c9616ed0dce2cf3/qa; will attempt to use it 2023-10-16T08:24:19.925 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_mchangir_ceph_6d9e81bfe885e1cd43cce3f41c9616ed0dce2cf3/qa/tasks 2023-10-16T08:24:19.925 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-16T08:24:19.927 INFO:teuthology.task.internal:Checking packages... 2023-10-16T08:24:19.952 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash 'c7471068426370a3754c2250e7729280233d552b' 2023-10-16T08:24:19.952 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-16T08:24:19.953 INFO:teuthology.packaging:ref: None 2023-10-16T08:24:19.953 INFO:teuthology.packaging:tag: None 2023-10-16T08:24:19.954 INFO:teuthology.packaging:branch: wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments 2023-10-16T08:24:19.954 INFO:teuthology.packaging:sha1: c7471068426370a3754c2250e7729280233d552b 2023-10-16T08:24:19.954 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments 2023-10-16T08:24:20.239 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-6675-gc7471068-1jammy 2023-10-16T08:24:20.240 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-16T08:24:20.249 INFO:teuthology.task.internal:no buildpackages task found 2023-10-16T08:24:20.249 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-16T08:24:20.280 INFO:teuthology.task.internal:Saving configuration 2023-10-16T08:24:20.299 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-16T08:24:20.313 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-16T08:24:20.340 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-10-16_08:10:12-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428951', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 08:18:54.782129', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIB3ZzL6r+rmuXciziG+tYxStBWPE3Kh4vulgXWhPBJAM'} 2023-10-16T08:24:20.362 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-10-16_08:10:12-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428951', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 08:18:54.783752', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIFt2ld1aJ1lVfXf3YR+FIm9iLqSiIsXaJ7ixgz8yecvj'} 2023-10-16T08:24:20.362 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-16T08:24:20.373 INFO:teuthology.task.internal:roles: ubuntu@smithi040.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-16T08:24:20.374 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-16T08:24:20.374 INFO:teuthology.run_tasks:Running task console_log... 2023-10-16T08:24:20.466 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fd71217b9d0>, signals=[15]) 2023-10-16T08:24:20.467 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-16T08:24:20.478 INFO:teuthology.task.internal:Opening connections... 2023-10-16T08:24:20.478 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi040.front.sepia.ceph.com 2023-10-16T08:24:20.480 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:24:20.557 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2023-10-16T08:24:20.558 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:24:20.632 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-16T08:24:20.641 DEBUG:teuthology.orchestra.run.smithi040:> uname -m 2023-10-16T08:24:20.647 INFO:teuthology.orchestra.run.smithi040.stdout:x86_64 2023-10-16T08:24:20.648 DEBUG:teuthology.orchestra.run.smithi040:> cat /etc/os-release 2023-10-16T08:24:20.698 INFO:teuthology.orchestra.run.smithi040.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-10-16T08:24:20.698 INFO:teuthology.orchestra.run.smithi040.stdout:NAME="Ubuntu" 2023-10-16T08:24:20.698 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_ID="22.04" 2023-10-16T08:24:20.698 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-10-16T08:24:20.699 INFO:teuthology.orchestra.run.smithi040.stdout:VERSION_CODENAME=jammy 2023-10-16T08:24:20.699 INFO:teuthology.orchestra.run.smithi040.stdout:ID=ubuntu 2023-10-16T08:24:20.699 INFO:teuthology.orchestra.run.smithi040.stdout:ID_LIKE=debian 2023-10-16T08:24:20.699 INFO:teuthology.orchestra.run.smithi040.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-16T08:24:20.700 INFO:teuthology.orchestra.run.smithi040.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-16T08:24:20.700 INFO:teuthology.orchestra.run.smithi040.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-16T08:24:20.700 INFO:teuthology.orchestra.run.smithi040.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-16T08:24:20.700 INFO:teuthology.orchestra.run.smithi040.stdout:UBUNTU_CODENAME=jammy 2023-10-16T08:24:20.701 INFO:teuthology.lock.ops:Updating smithi040.front.sepia.ceph.com on lock server 2023-10-16T08:24:20.723 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2023-10-16T08:24:20.729 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2023-10-16T08:24:20.729 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2023-10-16T08:24:20.773 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-10-16T08:24:20.774 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="Ubuntu" 2023-10-16T08:24:20.774 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="22.04" 2023-10-16T08:24:20.774 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-10-16T08:24:20.774 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_CODENAME=jammy 2023-10-16T08:24:20.775 INFO:teuthology.orchestra.run.smithi086.stdout:ID=ubuntu 2023-10-16T08:24:20.775 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE=debian 2023-10-16T08:24:20.775 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://www.ubuntu.com/" 2023-10-16T08:24:20.775 INFO:teuthology.orchestra.run.smithi086.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-10-16T08:24:20.775 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-10-16T08:24:20.776 INFO:teuthology.orchestra.run.smithi086.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-10-16T08:24:20.776 INFO:teuthology.orchestra.run.smithi086.stdout:UBUNTU_CODENAME=jammy 2023-10-16T08:24:20.776 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2023-10-16T08:24:20.798 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-16T08:24:20.808 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-16T08:24:20.868 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-16T08:24:20.869 DEBUG:teuthology.orchestra.run.smithi040:> test '!' -e /home/ubuntu/cephtest 2023-10-16T08:24:20.872 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2023-10-16T08:24:20.876 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-16T08:24:20.901 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-16T08:24:20.902 DEBUG:teuthology.orchestra.run.smithi040:> test -z $(ls -A /var/lib/ceph) 2023-10-16T08:24:20.918 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2023-10-16T08:24:20.934 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-16T08:24:20.991 INFO:teuthology.run_tasks:Running task kernel... 2023-10-16T08:24:21.006 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-16T08:24:21.007 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-16T08:24:21.007 DEBUG:teuthology.orchestra.run.smithi040:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-16T08:24:21.008 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-16T08:24:21.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:24:21.013 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:24:21.014 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2023-10-16T08:24:21.015 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2023-10-16T08:24:21.061 INFO:teuthology.orchestra.run.smithi040.stdout:5.15.0-56-generic 2023-10-16T08:24:21.061 INFO:teuthology.task.kernel:Running kernel on smithi040: 5.15.0-56-generic 2023-10-16T08:24:21.062 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get clean 2023-10-16T08:24:21.063 INFO:teuthology.orchestra.run.smithi086.stdout:5.15.0-56-generic 2023-10-16T08:24:21.063 INFO:teuthology.task.kernel:Running kernel on smithi086: 5.15.0-56-generic 2023-10-16T08:24:21.063 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-get clean 2023-10-16T08:24:21.186 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get update 2023-10-16T08:24:21.201 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-get update 2023-10-16T08:24:21.408 INFO:teuthology.orchestra.run.smithi040.stdout:Get:1 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2023-10-16T08:24:21.412 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-16T08:24:21.433 INFO:teuthology.orchestra.run.smithi086.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2023-10-16T08:24:21.435 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 http://security.ubuntu.com/ubuntu jammy-security InRelease [110 kB] 2023-10-16T08:24:21.492 INFO:teuthology.orchestra.run.smithi086.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2023-10-16T08:24:21.527 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-16T08:24:21.609 INFO:teuthology.orchestra.run.smithi040.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease [119 kB] 2023-10-16T08:24:21.624 INFO:teuthology.orchestra.run.smithi040.stdout:Get:4 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [346 kB] 2023-10-16T08:24:21.647 INFO:teuthology.orchestra.run.smithi086.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,062 kB] 2023-10-16T08:24:21.714 INFO:teuthology.orchestra.run.smithi040.stdout:Get:5 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [856 kB] 2023-10-16T08:24:21.728 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [506 kB] 2023-10-16T08:24:21.735 INFO:teuthology.orchestra.run.smithi086.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [234 kB] 2023-10-16T08:24:21.738 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.0 kB] 2023-10-16T08:24:21.739 INFO:teuthology.orchestra.run.smithi086.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [32.3 kB] 2023-10-16T08:24:21.739 INFO:teuthology.orchestra.run.smithi086.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [974 kB] 2023-10-16T08:24:21.742 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [175 kB] 2023-10-16T08:24:21.747 INFO:teuthology.orchestra.run.smithi040.stdout:Get:7 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2023-10-16T08:24:21.747 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [32.0 kB] 2023-10-16T08:24:21.748 INFO:teuthology.orchestra.run.smithi040.stdout:Get:9 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [953 kB] 2023-10-16T08:24:21.751 INFO:teuthology.orchestra.run.smithi086.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [157 kB] 2023-10-16T08:24:21.760 INFO:teuthology.orchestra.run.smithi040.stdout:Get:10 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [154 kB] 2023-10-16T08:24:21.762 INFO:teuthology.orchestra.run.smithi040.stdout:Get:11 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [532 B] 2023-10-16T08:24:21.762 INFO:teuthology.orchestra.run.smithi040.stdout:Get:12 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [789 kB] 2023-10-16T08:24:21.769 INFO:teuthology.orchestra.run.smithi086.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [532 B] 2023-10-16T08:24:21.769 INFO:teuthology.orchestra.run.smithi086.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [658 kB] 2023-10-16T08:24:21.770 INFO:teuthology.orchestra.run.smithi040.stdout:Get:13 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [560 kB] 2023-10-16T08:24:21.776 INFO:teuthology.orchestra.run.smithi040.stdout:Get:14 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [145 kB] 2023-10-16T08:24:21.776 INFO:teuthology.orchestra.run.smithi086.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [991 kB] 2023-10-16T08:24:21.777 INFO:teuthology.orchestra.run.smithi040.stdout:Get:15 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.7 kB] 2023-10-16T08:24:21.777 INFO:teuthology.orchestra.run.smithi040.stdout:Get:16 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,032 B] 2023-10-16T08:24:21.784 INFO:teuthology.orchestra.run.smithi040.stdout:Get:17 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [36.5 kB] 2023-10-16T08:24:21.785 INFO:teuthology.orchestra.run.smithi040.stdout:Get:18 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,060 B] 2023-10-16T08:24:21.786 INFO:teuthology.orchestra.run.smithi086.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [217 kB] 2023-10-16T08:24:21.789 INFO:teuthology.orchestra.run.smithi086.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.0 kB] 2023-10-16T08:24:21.789 INFO:teuthology.orchestra.run.smithi086.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [41.6 kB] 2023-10-16T08:24:21.790 INFO:teuthology.orchestra.run.smithi086.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [3,888 B] 2023-10-16T08:24:21.790 INFO:teuthology.orchestra.run.smithi086.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [9,768 B] 2023-10-16T08:24:21.791 INFO:teuthology.orchestra.run.smithi086.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2023-10-16T08:24:21.791 INFO:teuthology.orchestra.run.smithi086.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [33.9 kB] 2023-10-16T08:24:21.792 INFO:teuthology.orchestra.run.smithi086.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [41.7 kB] 2023-10-16T08:24:21.794 INFO:teuthology.orchestra.run.smithi040.stdout:Get:19 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2023-10-16T08:24:21.806 INFO:teuthology.orchestra.run.smithi086.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2023-10-16T08:24:21.806 INFO:teuthology.orchestra.run.smithi086.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2023-10-16T08:24:21.814 INFO:teuthology.orchestra.run.smithi086.stdout:Get:25 http://security.ubuntu.com/ubuntu jammy-security/main i386 Packages [346 kB] 2023-10-16T08:24:21.823 INFO:teuthology.orchestra.run.smithi086.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [24.3 kB] 2023-10-16T08:24:21.823 INFO:teuthology.orchestra.run.smithi086.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [13.4 kB] 2023-10-16T08:24:21.824 INFO:teuthology.orchestra.run.smithi086.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.4 kB] 2023-10-16T08:24:21.824 INFO:teuthology.orchestra.run.smithi086.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [640 B] 2023-10-16T08:24:21.916 INFO:teuthology.orchestra.run.smithi086.stdout:Get:30 http://security.ubuntu.com/ubuntu jammy-security/main amd64 Packages [856 kB] 2023-10-16T08:24:21.962 INFO:teuthology.orchestra.run.smithi040.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-backports InRelease [109 kB] 2023-10-16T08:24:21.977 INFO:teuthology.orchestra.run.smithi086.stdout:Get:31 http://security.ubuntu.com/ubuntu jammy-security/main Translation-en [175 kB] 2023-10-16T08:24:21.982 INFO:teuthology.orchestra.run.smithi086.stdout:Get:32 http://security.ubuntu.com/ubuntu jammy-security/main amd64 c-n-f Metadata [11.4 kB] 2023-10-16T08:24:21.983 INFO:teuthology.orchestra.run.smithi086.stdout:Get:33 http://security.ubuntu.com/ubuntu jammy-security/restricted i386 Packages [32.0 kB] 2023-10-16T08:24:21.984 INFO:teuthology.orchestra.run.smithi086.stdout:Get:34 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 Packages [953 kB] 2023-10-16T08:24:22.016 INFO:teuthology.orchestra.run.smithi086.stdout:Get:35 http://security.ubuntu.com/ubuntu jammy-security/restricted Translation-en [154 kB] 2023-10-16T08:24:22.019 INFO:teuthology.orchestra.run.smithi086.stdout:Get:36 http://security.ubuntu.com/ubuntu jammy-security/restricted amd64 c-n-f Metadata [532 B] 2023-10-16T08:24:22.020 INFO:teuthology.orchestra.run.smithi086.stdout:Get:37 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 Packages [789 kB] 2023-10-16T08:24:22.036 INFO:teuthology.orchestra.run.smithi086.stdout:Get:38 http://security.ubuntu.com/ubuntu jammy-security/universe i386 Packages [560 kB] 2023-10-16T08:24:22.047 INFO:teuthology.orchestra.run.smithi086.stdout:Get:39 http://security.ubuntu.com/ubuntu jammy-security/universe Translation-en [145 kB] 2023-10-16T08:24:22.050 INFO:teuthology.orchestra.run.smithi086.stdout:Get:40 http://security.ubuntu.com/ubuntu jammy-security/universe amd64 c-n-f Metadata [16.7 kB] 2023-10-16T08:24:22.056 INFO:teuthology.orchestra.run.smithi086.stdout:Get:41 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 Packages [36.5 kB] 2023-10-16T08:24:22.057 INFO:teuthology.orchestra.run.smithi086.stdout:Get:42 http://security.ubuntu.com/ubuntu jammy-security/multiverse i386 Packages [1,032 B] 2023-10-16T08:24:22.057 INFO:teuthology.orchestra.run.smithi086.stdout:Get:43 http://security.ubuntu.com/ubuntu jammy-security/multiverse Translation-en [7,060 B] 2023-10-16T08:24:22.087 INFO:teuthology.orchestra.run.smithi086.stdout:Get:44 http://security.ubuntu.com/ubuntu jammy-security/multiverse amd64 c-n-f Metadata [260 B] 2023-10-16T08:24:22.100 INFO:teuthology.orchestra.run.smithi040.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main i386 Packages [506 kB] 2023-10-16T08:24:22.283 INFO:teuthology.orchestra.run.smithi040.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 Packages [1,062 kB] 2023-10-16T08:24:22.408 INFO:teuthology.orchestra.run.smithi040.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy-updates/main Translation-en [234 kB] 2023-10-16T08:24:22.427 INFO:teuthology.orchestra.run.smithi040.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 c-n-f Metadata [16.0 kB] 2023-10-16T08:24:22.427 INFO:teuthology.orchestra.run.smithi040.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 Packages [974 kB] 2023-10-16T08:24:22.477 INFO:teuthology.orchestra.run.smithi040.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy-updates/restricted i386 Packages [32.3 kB] 2023-10-16T08:24:22.478 INFO:teuthology.orchestra.run.smithi040.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/restricted Translation-en [157 kB] 2023-10-16T08:24:22.484 INFO:teuthology.orchestra.run.smithi040.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/restricted amd64 c-n-f Metadata [532 B] 2023-10-16T08:24:22.649 INFO:teuthology.orchestra.run.smithi040.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 Packages [991 kB] 2023-10-16T08:24:23.269 INFO:teuthology.orchestra.run.smithi040.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/universe i386 Packages [658 kB] 2023-10-16T08:24:23.307 INFO:teuthology.orchestra.run.smithi040.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy-updates/universe Translation-en [217 kB] 2023-10-16T08:24:23.319 INFO:teuthology.orchestra.run.smithi040.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 c-n-f Metadata [22.0 kB] 2023-10-16T08:24:23.321 INFO:teuthology.orchestra.run.smithi040.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 Packages [41.6 kB] 2023-10-16T08:24:23.323 INFO:teuthology.orchestra.run.smithi040.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse i386 Packages [3,888 B] 2023-10-16T08:24:23.324 INFO:teuthology.orchestra.run.smithi040.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse Translation-en [9,768 B] 2023-10-16T08:24:23.324 INFO:teuthology.orchestra.run.smithi040.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy-updates/multiverse amd64 c-n-f Metadata [472 B] 2023-10-16T08:24:23.325 INFO:teuthology.orchestra.run.smithi040.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy-backports/main i386 Packages [33.9 kB] 2023-10-16T08:24:23.326 INFO:teuthology.orchestra.run.smithi040.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 Packages [41.7 kB] 2023-10-16T08:24:23.351 INFO:teuthology.orchestra.run.smithi040.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-backports/main Translation-en [10.5 kB] 2023-10-16T08:24:23.516 INFO:teuthology.orchestra.run.smithi040.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy-backports/main amd64 c-n-f Metadata [388 B] 2023-10-16T08:24:23.598 INFO:teuthology.orchestra.run.smithi040.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 Packages [24.3 kB] 2023-10-16T08:24:23.694 INFO:teuthology.orchestra.run.smithi040.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-backports/universe i386 Packages [13.4 kB] 2023-10-16T08:24:23.723 INFO:teuthology.orchestra.run.smithi040.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-backports/universe Translation-en [16.4 kB] 2023-10-16T08:24:23.774 INFO:teuthology.orchestra.run.smithi040.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy-backports/universe amd64 c-n-f Metadata [640 B] 2023-10-16T08:24:27.029 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 9,489 kB in 2s (6,309 kB/s) 2023-10-16T08:24:28.351 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2023-10-16T08:24:28.368 DEBUG:teuthology.orchestra.run.smithi086:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-16T08:24:28.378 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 9,489 kB in 3s (3,771 kB/s) 2023-10-16T08:24:28.474 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2023-10-16T08:24:28.658 INFO:teuthology.orchestra.run.smithi086.stdout:Building dependency tree... 2023-10-16T08:24:28.659 INFO:teuthology.orchestra.run.smithi086.stdout:Reading state information... 2023-10-16T08:24:28.816 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages were automatically installed and are no longer required: 2023-10-16T08:24:28.816 INFO:teuthology.orchestra.run.smithi086.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-16T08:24:28.816 INFO:teuthology.orchestra.run.smithi086.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-16T08:24:28.817 INFO:teuthology.orchestra.run.smithi086.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-16T08:24:28.817 INFO:teuthology.orchestra.run.smithi086.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2023-10-16T08:24:28.818 INFO:teuthology.orchestra.run.smithi086.stdout: python2.7-minimal 2023-10-16T08:24:28.818 INFO:teuthology.orchestra.run.smithi086.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-16T08:24:28.818 INFO:teuthology.orchestra.run.smithi086.stdout:The following additional packages will be installed: 2023-10-16T08:24:28.819 INFO:teuthology.orchestra.run.smithi086.stdout: linux-generic linux-headers-5.15.0-86 linux-headers-5.15.0-86-generic 2023-10-16T08:24:28.819 INFO:teuthology.orchestra.run.smithi086.stdout: linux-headers-generic linux-image-5.15.0-86-generic 2023-10-16T08:24:28.819 INFO:teuthology.orchestra.run.smithi086.stdout: linux-modules-5.15.0-86-generic linux-modules-extra-5.15.0-86-generic 2023-10-16T08:24:28.820 INFO:teuthology.orchestra.run.smithi086.stdout:Suggested packages: 2023-10-16T08:24:28.820 INFO:teuthology.orchestra.run.smithi086.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2023-10-16T08:24:28.820 INFO:teuthology.orchestra.run.smithi086.stdout:Recommended packages: 2023-10-16T08:24:28.820 INFO:teuthology.orchestra.run.smithi086.stdout: thermald 2023-10-16T08:24:28.869 INFO:teuthology.orchestra.run.smithi086.stdout:The following NEW packages will be installed: 2023-10-16T08:24:28.869 INFO:teuthology.orchestra.run.smithi086.stdout: linux-headers-5.15.0-86 linux-headers-5.15.0-86-generic 2023-10-16T08:24:28.869 INFO:teuthology.orchestra.run.smithi086.stdout: linux-image-5.15.0-86-generic linux-modules-5.15.0-86-generic 2023-10-16T08:24:28.870 INFO:teuthology.orchestra.run.smithi086.stdout: linux-modules-extra-5.15.0-86-generic 2023-10-16T08:24:28.870 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages will be upgraded: 2023-10-16T08:24:28.871 INFO:teuthology.orchestra.run.smithi086.stdout: linux-generic linux-headers-generic linux-image-generic 2023-10-16T08:24:28.930 INFO:teuthology.orchestra.run.smithi086.stdout:3 upgraded, 5 newly installed, 0 to remove and 268 not upgraded. 2023-10-16T08:24:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:Need to get 113 MB of archives. 2023-10-16T08:24:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:After this operation, 582 MB of additional disk space will be used. 2023-10-16T08:24:28.931 INFO:teuthology.orchestra.run.smithi086.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-86-generic amd64 5.15.0-86.96 [22.4 MB] 2023-10-16T08:24:29.523 INFO:teuthology.orchestra.run.smithi086.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-86-generic amd64 5.15.0-86.96 [11.5 MB] 2023-10-16T08:24:29.588 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2023-10-16T08:24:29.605 DEBUG:teuthology.orchestra.run.smithi040:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-16T08:24:29.667 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2023-10-16T08:24:29.699 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-86-generic amd64 5.15.0-86.96 [63.9 MB] 2023-10-16T08:24:29.854 INFO:teuthology.orchestra.run.smithi040.stdout:Building dependency tree... 2023-10-16T08:24:29.855 INFO:teuthology.orchestra.run.smithi040.stdout:Reading state information... 2023-10-16T08:24:30.018 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages were automatically installed and are no longer required: 2023-10-16T08:24:30.019 INFO:teuthology.orchestra.run.smithi040.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-16T08:24:30.019 INFO:teuthology.orchestra.run.smithi040.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-16T08:24:30.020 INFO:teuthology.orchestra.run.smithi040.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-16T08:24:30.020 INFO:teuthology.orchestra.run.smithi040.stdout: python2 python2-dev python2-minimal python2.7 python2.7-dev 2023-10-16T08:24:30.020 INFO:teuthology.orchestra.run.smithi040.stdout: python2.7-minimal 2023-10-16T08:24:30.020 INFO:teuthology.orchestra.run.smithi040.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-16T08:24:30.021 INFO:teuthology.orchestra.run.smithi040.stdout:The following additional packages will be installed: 2023-10-16T08:24:30.022 INFO:teuthology.orchestra.run.smithi040.stdout: linux-generic linux-headers-5.15.0-86 linux-headers-5.15.0-86-generic 2023-10-16T08:24:30.022 INFO:teuthology.orchestra.run.smithi040.stdout: linux-headers-generic linux-image-5.15.0-86-generic 2023-10-16T08:24:30.022 INFO:teuthology.orchestra.run.smithi040.stdout: linux-modules-5.15.0-86-generic linux-modules-extra-5.15.0-86-generic 2023-10-16T08:24:30.023 INFO:teuthology.orchestra.run.smithi040.stdout:Suggested packages: 2023-10-16T08:24:30.023 INFO:teuthology.orchestra.run.smithi040.stdout: fdutils linux-doc | linux-source-5.15.0 linux-tools 2023-10-16T08:24:30.023 INFO:teuthology.orchestra.run.smithi040.stdout:Recommended packages: 2023-10-16T08:24:30.023 INFO:teuthology.orchestra.run.smithi040.stdout: thermald 2023-10-16T08:24:30.071 INFO:teuthology.orchestra.run.smithi040.stdout:The following NEW packages will be installed: 2023-10-16T08:24:30.072 INFO:teuthology.orchestra.run.smithi040.stdout: linux-headers-5.15.0-86 linux-headers-5.15.0-86-generic 2023-10-16T08:24:30.072 INFO:teuthology.orchestra.run.smithi040.stdout: linux-image-5.15.0-86-generic linux-modules-5.15.0-86-generic 2023-10-16T08:24:30.072 INFO:teuthology.orchestra.run.smithi040.stdout: linux-modules-extra-5.15.0-86-generic 2023-10-16T08:24:30.074 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages will be upgraded: 2023-10-16T08:24:30.074 INFO:teuthology.orchestra.run.smithi040.stdout: linux-generic linux-headers-generic linux-image-generic 2023-10-16T08:24:30.254 INFO:teuthology.orchestra.run.smithi040.stdout:3 upgraded, 5 newly installed, 0 to remove and 268 not upgraded. 2023-10-16T08:24:30.255 INFO:teuthology.orchestra.run.smithi040.stdout:Need to get 113 MB of archives. 2023-10-16T08:24:30.255 INFO:teuthology.orchestra.run.smithi040.stdout:After this operation, 582 MB of additional disk space will be used. 2023-10-16T08:24:30.256 INFO:teuthology.orchestra.run.smithi040.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-5.15.0-86-generic amd64 5.15.0-86.96 [22.4 MB] 2023-10-16T08:24:31.219 INFO:teuthology.orchestra.run.smithi086.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.86.83 [1,702 B] 2023-10-16T08:24:31.219 INFO:teuthology.orchestra.run.smithi086.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.86.83 [2,472 B] 2023-10-16T08:24:31.219 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-86 all 5.15.0-86.96 [12.3 MB] 2023-10-16T08:24:31.606 INFO:teuthology.orchestra.run.smithi086.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-86-generic amd64 5.15.0-86.96 [2,882 kB] 2023-10-16T08:24:31.649 INFO:teuthology.orchestra.run.smithi040.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-5.15.0-86-generic amd64 5.15.0-86.96 [11.5 MB] 2023-10-16T08:24:31.716 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.86.83 [2,328 B] 2023-10-16T08:24:32.003 INFO:teuthology.orchestra.run.smithi040.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-modules-extra-5.15.0-86-generic amd64 5.15.0-86.96 [63.9 MB] 2023-10-16T08:24:32.111 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 113 MB in 3s (39.8 MB/s) 2023-10-16T08:24:32.302 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package linux-modules-5.15.0-86-generic. 2023-10-16T08:24:34.257 INFO:teuthology.orchestra.run.smithi040.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-generic amd64 5.15.0.86.83 [1,702 B] 2023-10-16T08:24:34.257 INFO:teuthology.orchestra.run.smithi040.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-image-generic amd64 5.15.0.86.83 [2,472 B] 2023-10-16T08:24:34.258 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-86 all 5.15.0-86.96 [12.3 MB] 2023-10-16T08:24:34.268 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2023-10-16T08:24:34.273 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../0-linux-modules-5.15.0-86-generic_5.15.0-86.96_amd64.deb ... 2023-10-16T08:24:34.399 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-modules-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-16T08:24:34.729 INFO:teuthology.orchestra.run.smithi040.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-5.15.0-86-generic amd64 5.15.0-86.96 [2,882 kB] 2023-10-16T08:24:34.786 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 linux-headers-generic amd64 5.15.0.86.83 [2,328 B] 2023-10-16T08:24:35.166 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 113 MB in 5s (24.0 MB/s) 2023-10-16T08:24:35.346 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-modules-5.15.0-86-generic. 2023-10-16T08:24:36.583 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package linux-image-5.15.0-86-generic. 2023-10-16T08:24:36.604 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../1-linux-image-5.15.0-86-generic_5.15.0-86.96_amd64.deb ... 2023-10-16T08:24:36.694 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-image-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-16T08:24:37.331 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-86-generic. 2023-10-16T08:24:37.352 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-86-generic_5.15.0-86.96_amd64.deb ... 2023-10-16T08:24:37.367 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 129542 files and directories currently installed.) 2023-10-16T08:24:37.371 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../0-linux-modules-5.15.0-86-generic_5.15.0-86.96_amd64.deb ... 2023-10-16T08:24:37.435 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-modules-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-16T08:24:37.594 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-modules-extra-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-16T08:24:39.853 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-image-5.15.0-86-generic. 2023-10-16T08:24:39.875 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../1-linux-image-5.15.0-86-generic_5.15.0-86.96_amd64.deb ... 2023-10-16T08:24:39.962 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-image-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-16T08:24:40.398 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-modules-extra-5.15.0-86-generic. 2023-10-16T08:24:40.419 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../2-linux-modules-extra-5.15.0-86-generic_5.15.0-86.96_amd64.deb ... 2023-10-16T08:24:40.456 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-modules-extra-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-16T08:24:44.069 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../3-linux-generic_5.15.0.86.83_amd64.deb ... 2023-10-16T08:24:44.188 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-generic (5.15.0.86.83) over (5.15.0.56.54) ... 2023-10-16T08:24:44.537 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.86.83_amd64.deb ... 2023-10-16T08:24:44.658 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-image-generic (5.15.0.86.83) over (5.15.0.56.54) ... 2023-10-16T08:24:44.986 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package linux-headers-5.15.0-86. 2023-10-16T08:24:45.009 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../5-linux-headers-5.15.0-86_5.15.0-86.96_all.deb ... 2023-10-16T08:24:45.044 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-headers-5.15.0-86 (5.15.0-86.96) ... 2023-10-16T08:24:46.825 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../3-linux-generic_5.15.0.86.83_amd64.deb ... 2023-10-16T08:24:46.952 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-generic (5.15.0.86.83) over (5.15.0.56.54) ... 2023-10-16T08:24:47.277 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../4-linux-image-generic_5.15.0.86.83_amd64.deb ... 2023-10-16T08:24:47.405 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-image-generic (5.15.0.86.83) over (5.15.0.56.54) ... 2023-10-16T08:24:47.717 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-headers-5.15.0-86. 2023-10-16T08:24:47.740 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../5-linux-headers-5.15.0-86_5.15.0-86.96_all.deb ... 2023-10-16T08:24:47.775 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-headers-5.15.0-86 (5.15.0-86.96) ... 2023-10-16T08:24:50.972 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package linux-headers-5.15.0-86-generic. 2023-10-16T08:24:51.009 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../6-linux-headers-5.15.0-86-generic_5.15.0-86.96_amd64.deb ... 2023-10-16T08:24:51.047 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-headers-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-16T08:24:53.488 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package linux-headers-5.15.0-86-generic. 2023-10-16T08:24:53.507 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../6-linux-headers-5.15.0-86-generic_5.15.0-86.96_amd64.deb ... 2023-10-16T08:24:53.546 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-headers-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-16T08:24:53.872 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.86.83_amd64.deb ... 2023-10-16T08:24:53.998 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking linux-headers-generic (5.15.0.86.83) over (5.15.0.56.54) ... 2023-10-16T08:24:54.373 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-headers-5.15.0-86 (5.15.0-86.96) ... 2023-10-16T08:24:54.494 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-headers-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-16T08:24:54.645 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-headers-generic (5.15.0.86.83) ... 2023-10-16T08:24:54.772 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-image-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-16T08:24:56.098 INFO:teuthology.orchestra.run.smithi086.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2023-10-16T08:24:56.098 INFO:teuthology.orchestra.run.smithi086.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2023-10-16T08:24:56.098 INFO:teuthology.orchestra.run.smithi086.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-86-generic 2023-10-16T08:24:56.098 INFO:teuthology.orchestra.run.smithi086.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-86-generic 2023-10-16T08:24:56.219 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../7-linux-headers-generic_5.15.0.86.83_amd64.deb ... 2023-10-16T08:24:56.268 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-modules-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-16T08:24:56.341 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking linux-headers-generic (5.15.0.86.83) over (5.15.0.56.54) ... 2023-10-16T08:24:56.658 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-headers-5.15.0-86 (5.15.0-86.96) ... 2023-10-16T08:24:56.779 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-headers-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-16T08:24:56.964 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-headers-generic (5.15.0.86.83) ... 2023-10-16T08:24:57.474 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-image-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-16T08:24:57.760 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-modules-extra-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-16T08:24:58.834 INFO:teuthology.orchestra.run.smithi040.stdout:I: /vmlinuz.old is now a symlink to boot/vmlinuz-5.15.0-56-generic 2023-10-16T08:24:58.835 INFO:teuthology.orchestra.run.smithi040.stdout:I: /initrd.img.old is now a symlink to boot/initrd.img-5.15.0-56-generic 2023-10-16T08:24:58.835 INFO:teuthology.orchestra.run.smithi040.stdout:I: /vmlinuz is now a symlink to boot/vmlinuz-5.15.0-86-generic 2023-10-16T08:24:58.835 INFO:teuthology.orchestra.run.smithi040.stdout:I: /initrd.img is now a symlink to boot/initrd.img-5.15.0-86-generic 2023-10-16T08:24:58.987 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-modules-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-16T08:24:59.203 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-image-generic (5.15.0.86.83) ... 2023-10-16T08:24:59.321 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up linux-generic (5.15.0.86.83) ... 2023-10-16T08:24:59.422 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for linux-image-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-16T08:24:59.485 INFO:teuthology.orchestra.run.smithi086.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-10-16T08:24:59.486 INFO:teuthology.orchestra.run.smithi086.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-86-generic 2023-10-16T08:25:00.404 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-modules-extra-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-16T08:25:01.797 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-image-generic (5.15.0.86.83) ... 2023-10-16T08:25:01.915 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up linux-generic (5.15.0.86.83) ... 2023-10-16T08:25:02.050 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for linux-image-5.15.0-86-generic (5.15.0-86.96) ... 2023-10-16T08:25:02.115 INFO:teuthology.orchestra.run.smithi040.stdout:/etc/kernel/postinst.d/initramfs-tools: 2023-10-16T08:25:02.115 INFO:teuthology.orchestra.run.smithi040.stdout:update-initramfs: Generating /boot/initrd.img-5.15.0-86-generic 2023-10-16T08:25:13.313 INFO:teuthology.orchestra.run.smithi086.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-10-16T08:25:13.314 INFO:teuthology.orchestra.run.smithi086.stdout:Sourcing file `/etc/default/grub' 2023-10-16T08:25:13.340 INFO:teuthology.orchestra.run.smithi086.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-16T08:25:13.368 INFO:teuthology.orchestra.run.smithi086.stdout:Generating grub configuration file ... 2023-10-16T08:25:13.683 INFO:teuthology.orchestra.run.smithi086.stdout:Found linux image: /boot/vmlinuz-5.15.0-86-generic 2023-10-16T08:25:13.711 INFO:teuthology.orchestra.run.smithi086.stdout:Found initrd image: /boot/initrd.img-5.15.0-86-generic 2023-10-16T08:25:14.047 INFO:teuthology.orchestra.run.smithi086.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-16T08:25:14.053 INFO:teuthology.orchestra.run.smithi086.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-16T08:25:14.124 INFO:teuthology.orchestra.run.smithi086.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-16T08:25:14.130 INFO:teuthology.orchestra.run.smithi086.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-16T08:25:14.312 INFO:teuthology.orchestra.run.smithi086.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-16T08:25:14.313 INFO:teuthology.orchestra.run.smithi086.stdout:Systems on them will not be added to the GRUB boot configuration. 2023-10-16T08:25:14.313 INFO:teuthology.orchestra.run.smithi086.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-16T08:25:14.355 INFO:teuthology.orchestra.run.smithi086.stdout:done 2023-10-16T08:25:14.827 DEBUG:teuthology.orchestra.run.smithi086:> dpkg -s linux-image-generic 2023-10-16T08:25:14.854 INFO:teuthology.orchestra.run.smithi086.stdout:Package: linux-image-generic 2023-10-16T08:25:14.854 INFO:teuthology.orchestra.run.smithi086.stdout:Status: install ok installed 2023-10-16T08:25:14.854 INFO:teuthology.orchestra.run.smithi086.stdout:Priority: optional 2023-10-16T08:25:14.854 INFO:teuthology.orchestra.run.smithi086.stdout:Section: kernel 2023-10-16T08:25:14.855 INFO:teuthology.orchestra.run.smithi086.stdout:Installed-Size: 20 2023-10-16T08:25:14.855 INFO:teuthology.orchestra.run.smithi086.stdout:Maintainer: Ubuntu Kernel Team 2023-10-16T08:25:14.855 INFO:teuthology.orchestra.run.smithi086.stdout:Architecture: amd64 2023-10-16T08:25:14.855 INFO:teuthology.orchestra.run.smithi086.stdout:Source: linux-meta 2023-10-16T08:25:14.855 INFO:teuthology.orchestra.run.smithi086.stdout:Version: 5.15.0.86.83 2023-10-16T08:25:14.856 INFO:teuthology.orchestra.run.smithi086.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-86), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-10-16T08:25:14.856 INFO:teuthology.orchestra.run.smithi086.stdout:Depends: linux-image-5.15.0-86-generic, linux-modules-extra-5.15.0-86-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-16T08:25:14.856 INFO:teuthology.orchestra.run.smithi086.stdout:Recommends: thermald 2023-10-16T08:25:14.856 INFO:teuthology.orchestra.run.smithi086.stdout:Description: Generic Linux kernel image 2023-10-16T08:25:14.856 INFO:teuthology.orchestra.run.smithi086.stdout: This package will always depend on the latest generic kernel image 2023-10-16T08:25:14.857 INFO:teuthology.orchestra.run.smithi086.stdout: available. 2023-10-16T08:25:14.857 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-86-generic 2023-10-16T08:25:14.857 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-16T08:25:14.857 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-16T08:25:14.858 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-10-16T08:25:14.858 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi086.front.sepia.ceph.com, path=None, version=distro) 2023-10-16T08:25:14.858 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-get clean 2023-10-16T08:25:15.010 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-get update 2023-10-16T08:25:15.152 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-16T08:25:15.152 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:2 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-16T08:25:15.165 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-10-16T08:25:15.185 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-10-16T08:25:16.401 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2023-10-16T08:25:16.419 DEBUG:teuthology.orchestra.run.smithi086:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-16T08:25:16.479 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2023-10-16T08:25:16.553 INFO:teuthology.orchestra.run.smithi040.stdout:/etc/kernel/postinst.d/zz-update-grub: 2023-10-16T08:25:16.554 INFO:teuthology.orchestra.run.smithi040.stdout:Sourcing file `/etc/default/grub' 2023-10-16T08:25:16.560 INFO:teuthology.orchestra.run.smithi040.stdout:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-16T08:25:16.588 INFO:teuthology.orchestra.run.smithi040.stdout:Generating grub configuration file ... 2023-10-16T08:25:16.664 INFO:teuthology.orchestra.run.smithi086.stdout:Building dependency tree... 2023-10-16T08:25:16.666 INFO:teuthology.orchestra.run.smithi086.stdout:Reading state information... 2023-10-16T08:25:16.826 INFO:teuthology.orchestra.run.smithi086.stdout:linux-image-generic is already the newest version (5.15.0.86.83). 2023-10-16T08:25:16.827 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages were automatically installed and are no longer required: 2023-10-16T08:25:16.827 INFO:teuthology.orchestra.run.smithi086.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-16T08:25:16.827 INFO:teuthology.orchestra.run.smithi086.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-16T08:25:16.827 INFO:teuthology.orchestra.run.smithi086.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-16T08:25:16.828 INFO:teuthology.orchestra.run.smithi086.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-16T08:25:16.828 INFO:teuthology.orchestra.run.smithi086.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-16T08:25:16.828 INFO:teuthology.orchestra.run.smithi086.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-16T08:25:16.866 INFO:teuthology.orchestra.run.smithi086.stdout:0 upgraded, 0 newly installed, 0 to remove and 268 not upgraded. 2023-10-16T08:25:16.868 DEBUG:teuthology.orchestra.run.smithi086:> dpkg -s linux-image-generic 2023-10-16T08:25:16.883 INFO:teuthology.orchestra.run.smithi086.stdout:Package: linux-image-generic 2023-10-16T08:25:16.884 INFO:teuthology.orchestra.run.smithi086.stdout:Status: install ok installed 2023-10-16T08:25:16.884 INFO:teuthology.orchestra.run.smithi086.stdout:Priority: optional 2023-10-16T08:25:16.884 INFO:teuthology.orchestra.run.smithi086.stdout:Section: kernel 2023-10-16T08:25:16.885 INFO:teuthology.orchestra.run.smithi086.stdout:Installed-Size: 20 2023-10-16T08:25:16.885 INFO:teuthology.orchestra.run.smithi086.stdout:Maintainer: Ubuntu Kernel Team 2023-10-16T08:25:16.885 INFO:teuthology.orchestra.run.smithi086.stdout:Architecture: amd64 2023-10-16T08:25:16.885 INFO:teuthology.orchestra.run.smithi086.stdout:Source: linux-meta 2023-10-16T08:25:16.886 INFO:teuthology.orchestra.run.smithi086.stdout:Version: 5.15.0.86.83 2023-10-16T08:25:16.886 INFO:teuthology.orchestra.run.smithi086.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-86), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-10-16T08:25:16.886 INFO:teuthology.orchestra.run.smithi086.stdout:Depends: linux-image-5.15.0-86-generic, linux-modules-extra-5.15.0-86-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-16T08:25:16.887 INFO:teuthology.orchestra.run.smithi086.stdout:Recommends: thermald 2023-10-16T08:25:16.887 INFO:teuthology.orchestra.run.smithi086.stdout:Description: Generic Linux kernel image 2023-10-16T08:25:16.887 INFO:teuthology.orchestra.run.smithi086.stdout: This package will always depend on the latest generic kernel image 2023-10-16T08:25:16.887 INFO:teuthology.orchestra.run.smithi086.stdout: available. 2023-10-16T08:25:16.888 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2023-10-16T08:25:16.929 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.oQmssYB3tj 2023-10-16T08:25:16.929 DEBUG:teuthology.orchestra.run.smithi086:> sudo cp /boot/grub/grub.cfg /tmp/tmp.oQmssYB3tj 2023-10-16T08:25:16.958 INFO:teuthology.orchestra.run.smithi040.stdout:Found linux image: /boot/vmlinuz-5.15.0-86-generic 2023-10-16T08:25:16.982 INFO:teuthology.orchestra.run.smithi040.stdout:Found initrd image: /boot/initrd.img-5.15.0-86-generic 2023-10-16T08:25:16.991 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.oQmssYB3tj 2023-10-16T08:25:17.101 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.oQmssYB3tj is 10KB 2023-10-16T08:25:17.116 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.oQmssYB3tj 2023-10-16T08:25:17.122 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-10-16T08:25:17.176 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:25:17.176 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-10-16T08:25:17.177 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-10-16T08:25:17.249 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-86-generic 2023-10-16T08:25:17.250 DEBUG:teuthology.orchestra.run.smithi086:> sudo update-grub 2023-10-16T08:25:17.331 INFO:teuthology.orchestra.run.smithi040.stdout:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-16T08:25:17.337 INFO:teuthology.orchestra.run.smithi040.stdout:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-16T08:25:17.406 INFO:teuthology.orchestra.run.smithi040.stdout:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-16T08:25:17.412 INFO:teuthology.orchestra.run.smithi040.stdout:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-16T08:25:17.560 INFO:teuthology.orchestra.run.smithi040.stdout:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-16T08:25:17.560 INFO:teuthology.orchestra.run.smithi040.stdout:Systems on them will not be added to the GRUB boot configuration. 2023-10-16T08:25:17.560 INFO:teuthology.orchestra.run.smithi040.stdout:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-16T08:25:17.603 INFO:teuthology.orchestra.run.smithi040.stdout:done 2023-10-16T08:25:18.118 DEBUG:teuthology.orchestra.run.smithi040:> dpkg -s linux-image-generic 2023-10-16T08:25:18.143 INFO:teuthology.orchestra.run.smithi040.stdout:Package: linux-image-generic 2023-10-16T08:25:18.143 INFO:teuthology.orchestra.run.smithi040.stdout:Status: install ok installed 2023-10-16T08:25:18.143 INFO:teuthology.orchestra.run.smithi040.stdout:Priority: optional 2023-10-16T08:25:18.143 INFO:teuthology.orchestra.run.smithi040.stdout:Section: kernel 2023-10-16T08:25:18.144 INFO:teuthology.orchestra.run.smithi040.stdout:Installed-Size: 20 2023-10-16T08:25:18.144 INFO:teuthology.orchestra.run.smithi040.stdout:Maintainer: Ubuntu Kernel Team 2023-10-16T08:25:18.144 INFO:teuthology.orchestra.run.smithi040.stdout:Architecture: amd64 2023-10-16T08:25:18.144 INFO:teuthology.orchestra.run.smithi040.stdout:Source: linux-meta 2023-10-16T08:25:18.144 INFO:teuthology.orchestra.run.smithi040.stdout:Version: 5.15.0.86.83 2023-10-16T08:25:18.145 INFO:teuthology.orchestra.run.smithi040.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-86), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-10-16T08:25:18.145 INFO:teuthology.orchestra.run.smithi040.stdout:Depends: linux-image-5.15.0-86-generic, linux-modules-extra-5.15.0-86-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-16T08:25:18.145 INFO:teuthology.orchestra.run.smithi040.stdout:Recommends: thermald 2023-10-16T08:25:18.145 INFO:teuthology.orchestra.run.smithi040.stdout:Description: Generic Linux kernel image 2023-10-16T08:25:18.145 INFO:teuthology.orchestra.run.smithi040.stdout: This package will always depend on the latest generic kernel image 2023-10-16T08:25:18.146 INFO:teuthology.orchestra.run.smithi040.stdout: available. 2023-10-16T08:25:18.147 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.15.0-56-generic Expected: 5.15.0-86-generic 2023-10-16T08:25:18.147 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-16T08:25:18.147 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-16T08:25:18.147 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-16T08:25:18.147 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi040.front.sepia.ceph.com, path=None, version=distro) 2023-10-16T08:25:18.148 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get clean 2023-10-16T08:25:18.319 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get update 2023-10-16T08:25:18.433 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-16T08:25:18.437 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-16T08:25:18.464 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-10-16T08:25:18.487 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-10-16T08:25:18.490 INFO:teuthology.orchestra.run.smithi086.stderr:Sourcing file `/etc/default/grub' 2023-10-16T08:25:18.491 INFO:teuthology.orchestra.run.smithi086.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-16T08:25:18.493 INFO:teuthology.orchestra.run.smithi086.stderr:Generating grub configuration file ... 2023-10-16T08:25:18.721 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.15.0-86-generic 2023-10-16T08:25:18.727 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.15.0-86-generic 2023-10-16T08:25:18.958 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-16T08:25:18.963 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-16T08:25:19.004 INFO:teuthology.orchestra.run.smithi086.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-16T08:25:19.009 INFO:teuthology.orchestra.run.smithi086.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-16T08:25:19.133 INFO:teuthology.orchestra.run.smithi086.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-16T08:25:19.133 INFO:teuthology.orchestra.run.smithi086.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-16T08:25:19.134 INFO:teuthology.orchestra.run.smithi086.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-16T08:25:19.170 INFO:teuthology.orchestra.run.smithi086.stderr:done 2023-10-16T08:25:19.173 DEBUG:teuthology.orchestra.run.smithi086:> sudo shutdown -r now 2023-10-16T08:25:19.751 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2023-10-16T08:25:19.768 DEBUG:teuthology.orchestra.run.smithi040:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y install linux-image-generic 2023-10-16T08:25:19.820 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2023-10-16T08:25:19.987 INFO:teuthology.orchestra.run.smithi040.stdout:Building dependency tree... 2023-10-16T08:25:19.988 INFO:teuthology.orchestra.run.smithi040.stdout:Reading state information... 2023-10-16T08:25:20.136 INFO:teuthology.orchestra.run.smithi040.stdout:linux-image-generic is already the newest version (5.15.0.86.83). 2023-10-16T08:25:20.137 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages were automatically installed and are no longer required: 2023-10-16T08:25:20.137 INFO:teuthology.orchestra.run.smithi040.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-16T08:25:20.137 INFO:teuthology.orchestra.run.smithi040.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-16T08:25:20.137 INFO:teuthology.orchestra.run.smithi040.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-16T08:25:20.138 INFO:teuthology.orchestra.run.smithi040.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-16T08:25:20.138 INFO:teuthology.orchestra.run.smithi040.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-16T08:25:20.138 INFO:teuthology.orchestra.run.smithi040.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-16T08:25:20.173 INFO:teuthology.orchestra.run.smithi040.stdout:0 upgraded, 0 newly installed, 0 to remove and 268 not upgraded. 2023-10-16T08:25:20.175 DEBUG:teuthology.orchestra.run.smithi040:> dpkg -s linux-image-generic 2023-10-16T08:25:20.232 INFO:teuthology.orchestra.run.smithi040.stdout:Package: linux-image-generic 2023-10-16T08:25:20.232 INFO:teuthology.orchestra.run.smithi040.stdout:Status: install ok installed 2023-10-16T08:25:20.232 INFO:teuthology.orchestra.run.smithi040.stdout:Priority: optional 2023-10-16T08:25:20.233 INFO:teuthology.orchestra.run.smithi040.stdout:Section: kernel 2023-10-16T08:25:20.233 INFO:teuthology.orchestra.run.smithi040.stdout:Installed-Size: 20 2023-10-16T08:25:20.233 INFO:teuthology.orchestra.run.smithi040.stdout:Maintainer: Ubuntu Kernel Team 2023-10-16T08:25:20.233 INFO:teuthology.orchestra.run.smithi040.stdout:Architecture: amd64 2023-10-16T08:25:20.233 INFO:teuthology.orchestra.run.smithi040.stdout:Source: linux-meta 2023-10-16T08:25:20.234 INFO:teuthology.orchestra.run.smithi040.stdout:Version: 5.15.0.86.83 2023-10-16T08:25:20.234 INFO:teuthology.orchestra.run.smithi040.stdout:Provides: spl-modules (= 2.1.5-1ubuntu6~22.04.1), v4l2loopback-modules (= 0.12.7-2ubuntu2~22.04.1), virtualbox-guest-modules (= 5.15.0-86), wireguard-modules (= 1.0.0), zfs-modules (= 2.1.5-1ubuntu6~22.04.1) 2023-10-16T08:25:20.234 INFO:teuthology.orchestra.run.smithi040.stdout:Depends: linux-image-5.15.0-86-generic, linux-modules-extra-5.15.0-86-generic, linux-firmware, intel-microcode, amd64-microcode 2023-10-16T08:25:20.234 INFO:teuthology.orchestra.run.smithi040.stdout:Recommends: thermald 2023-10-16T08:25:20.234 INFO:teuthology.orchestra.run.smithi040.stdout:Description: Generic Linux kernel image 2023-10-16T08:25:20.235 INFO:teuthology.orchestra.run.smithi040.stdout: This package will always depend on the latest generic kernel image 2023-10-16T08:25:20.235 INFO:teuthology.orchestra.run.smithi040.stdout: available. 2023-10-16T08:25:20.235 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2023-10-16T08:25:20.280 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.Ladnlkdmp1 2023-10-16T08:25:20.281 DEBUG:teuthology.orchestra.run.smithi040:> sudo cp /boot/grub/grub.cfg /tmp/tmp.Ladnlkdmp1 2023-10-16T08:25:20.337 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.Ladnlkdmp1 2023-10-16T08:25:20.457 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.Ladnlkdmp1 is 10KB 2023-10-16T08:25:20.470 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.Ladnlkdmp1 2023-10-16T08:25:20.475 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/grub.d/01_ceph_kernel 2023-10-16T08:25:20.529 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:25:20.530 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/grub.d/01_ceph_kernel 2023-10-16T08:25:20.530 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 755 /etc/grub.d/01_ceph_kernel 2023-10-16T08:25:20.596 INFO:teuthology.task.kernel:Distro Kernel Version: 5.15.0-86-generic 2023-10-16T08:25:20.597 DEBUG:teuthology.orchestra.run.smithi040:> sudo update-grub 2023-10-16T08:25:21.779 INFO:teuthology.orchestra.run.smithi040.stderr:Sourcing file `/etc/default/grub' 2023-10-16T08:25:21.781 INFO:teuthology.orchestra.run.smithi040.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-10-16T08:25:21.783 INFO:teuthology.orchestra.run.smithi040.stderr:Generating grub configuration file ... 2023-10-16T08:25:22.030 INFO:teuthology.orchestra.run.smithi040.stderr:Found linux image: /boot/vmlinuz-5.15.0-86-generic 2023-10-16T08:25:22.037 INFO:teuthology.orchestra.run.smithi040.stderr:Found initrd image: /boot/initrd.img-5.15.0-86-generic 2023-10-16T08:25:22.319 INFO:teuthology.orchestra.run.smithi040.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-10-16T08:25:22.324 INFO:teuthology.orchestra.run.smithi040.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-10-16T08:25:22.367 INFO:teuthology.orchestra.run.smithi040.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-10-16T08:25:22.373 INFO:teuthology.orchestra.run.smithi040.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-10-16T08:25:22.504 INFO:teuthology.orchestra.run.smithi040.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-10-16T08:25:22.504 INFO:teuthology.orchestra.run.smithi040.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-10-16T08:25:22.505 INFO:teuthology.orchestra.run.smithi040.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-10-16T08:25:22.527 INFO:teuthology.orchestra.run.smithi040.stderr:done 2023-10-16T08:25:22.536 DEBUG:teuthology.orchestra.run.smithi040:> sudo shutdown -r now 2023-10-16T08:25:49.202 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-16T08:25:49.203 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2023-10-16T08:25:49.204 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:25:52.539 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-16T08:25:52.540 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-10-16T08:25:52.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:26:07.610 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2023-10-16T08:26:10.942 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.40 2023-10-16T08:26:16.614 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2023-10-16T08:26:16.615 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:26:19.674 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.86 2023-10-16T08:26:19.944 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-10-16T08:26:19.945 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:26:23.002 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.40 2023-10-16T08:26:31.682 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2023-10-16T08:26:31.683 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:26:35.006 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-10-16T08:26:35.007 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:27:03.385 DEBUG:teuthology.orchestra.run.smithi086:> true 2023-10-16T08:27:03.987 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2023-10-16T08:27:03.988 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.15.0-86-generic"... 2023-10-16T08:27:03.988 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2023-10-16T08:27:04.034 INFO:teuthology.orchestra.run.smithi086.stdout:5.15.0-86-generic 2023-10-16T08:27:04.034 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-86-generic vs 5.15.0-86-generic 2023-10-16T08:27:04.034 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-16T08:27:04.035 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-16T08:27:05.036 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-16T08:27:05.036 DEBUG:teuthology.orchestra.run.smithi086:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-16T08:27:05.401 INFO:teuthology.orchestra.run.smithi086.stdout:ttyS1 2023-10-16T08:27:05.420 DEBUG:teuthology.parallel:result is None 2023-10-16T08:27:07.362 DEBUG:teuthology.orchestra.run.smithi040:> true 2023-10-16T08:27:08.449 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-10-16T08:27:08.449 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.15.0-86-generic"... 2023-10-16T08:27:08.450 DEBUG:teuthology.orchestra.run.smithi040:> uname -r 2023-10-16T08:27:08.496 INFO:teuthology.orchestra.run.smithi040.stdout:5.15.0-86-generic 2023-10-16T08:27:08.496 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-86-generic vs 5.15.0-86-generic 2023-10-16T08:27:08.496 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-16T08:27:08.496 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-10-16T08:27:09.497 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-16T08:27:09.498 DEBUG:teuthology.orchestra.run.smithi040:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-16T08:27:09.611 INFO:teuthology.orchestra.run.smithi040.stdout:ttyS1 2023-10-16T08:27:09.638 DEBUG:teuthology.parallel:result is None 2023-10-16T08:27:09.639 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-16T08:27:09.646 INFO:teuthology.task.internal:Creating test directory... 2023-10-16T08:27:09.647 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-16T08:27:09.650 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-16T08:27:09.656 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-16T08:27:09.683 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-16T08:27:09.733 INFO:teuthology.task.internal:Creating archive directory... 2023-10-16T08:27:09.734 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-16T08:27:09.737 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-16T08:27:09.762 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-16T08:27:09.783 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-16T08:27:09.784 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-16T08:27:09.789 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-16T08:27:09.818 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T08:27:09.819 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T08:27:09.825 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T08:27:09.828 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-16T08:27:09.830 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-16T08:27:09.837 INFO:teuthology.task.internal:Configuring sudo... 2023-10-16T08:27:09.837 DEBUG:teuthology.orchestra.run.smithi040:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-16T08:27:09.872 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-16T08:27:09.890 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-16T08:27:09.899 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-16T08:27:09.899 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-16T08:27:09.925 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-16T08:27:09.938 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-16T08:27:09.976 DEBUG:teuthology.orchestra.run.smithi040:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-16T08:27:10.024 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:27:10.024 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-16T08:27:10.191 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-16T08:27:10.197 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-16T08:27:10.242 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:27:10.242 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-16T08:27:10.314 DEBUG:teuthology.orchestra.run.smithi040:> sudo service rsyslog restart 2023-10-16T08:27:10.317 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2023-10-16T08:27:10.461 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-16T08:27:10.469 INFO:teuthology.task.internal:Starting timer... 2023-10-16T08:27:10.469 INFO:teuthology.run_tasks:Running task pcp... 2023-10-16T08:27:10.512 INFO:teuthology.run_tasks:Running task selinux... 2023-10-16T08:27:10.560 DEBUG:teuthology.task.selinux:Excluding smithi040: OS 'ubuntu' does not support SELinux 2023-10-16T08:27:10.561 DEBUG:teuthology.task.selinux:Excluding smithi086: OS 'ubuntu' does not support SELinux 2023-10-16T08:27:10.561 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-16T08:27:10.562 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-10-16T08:27:10.562 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-16T08:27:10.562 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-16T08:27:10.613 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-16T08:27:10.613 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-16T08:27:10.630 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-16T08:27:10.631 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi040.front.sepia.ceph.com,smithi086.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-16T08:34:54.606 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi040.front.sepia.ceph.com'), Remote(name='ubuntu@smithi086.front.sepia.ceph.com')] 2023-10-16T08:34:54.608 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-10-16T08:34:54.609 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi040.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:34:54.686 DEBUG:teuthology.orchestra.run.smithi040:> true 2023-10-16T08:34:54.802 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi040.front.sepia.ceph.com' 2023-10-16T08:34:54.803 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2023-10-16T08:34:54.804 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-16T08:34:54.882 DEBUG:teuthology.orchestra.run.smithi086:> true 2023-10-16T08:34:54.986 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2023-10-16T08:34:54.987 INFO:teuthology.run_tasks:Running task clock... 2023-10-16T08:34:54.998 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-16T08:34:54.999 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-16T08:34:55.000 DEBUG:teuthology.orchestra.run.smithi040:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T08:34:55.002 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-16T08:34:55.003 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T08:34:55.029 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:34:55 ntpd[27891]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-10-16T08:34:55.030 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:34:55 ntpd[27891]: Command line: ntpd -gq 2023-10-16T08:34:55.030 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:34:55 ntpd[27891]: ---------------------------------------------------- 2023-10-16T08:34:55.030 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:34:55 ntpd[27891]: ntp-4 is maintained by Network Time Foundation, 2023-10-16T08:34:55.030 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:34:55 ntpd[27891]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-10-16T08:34:55.031 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:34:55 ntpd[27891]: corporation. Support and training for ntp-4 are 2023-10-16T08:34:55.031 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:34:55 ntpd[27891]: available at https://www.nwtime.org/support 2023-10-16T08:34:55.031 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:34:55 ntpd[27891]: ---------------------------------------------------- 2023-10-16T08:34:55.032 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:34:55 ntpd[27891]: proto: precision = 0.045 usec (-24) 2023-10-16T08:34:55.032 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:34:55 ntpd[27891]: basedate set to 2022-02-04 2023-10-16T08:34:55.032 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:34:55 ntpd[27891]: gps base set to 2022-02-06 (week 2196) 2023-10-16T08:34:55.033 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:34:55 ntpd[27891]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-16T08:34:55.033 INFO:teuthology.orchestra.run.smithi040.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-16T08:34:55.033 INFO:teuthology.orchestra.run.smithi040.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-16T08:34:55.034 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:34:55 ntpd[27891]: restrict ::: KOD does nothing without LIMITED. 2023-10-16T08:34:55.035 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:34:55 ntpd[27891]: Listen and drop on 0 v6wildcard [::]:123 2023-10-16T08:34:55.035 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:34:55 ntpd[27891]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-16T08:34:55.035 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:34:55 ntpd[27891]: Listen normally on 2 lo 127.0.0.1:123 2023-10-16T08:34:55.035 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:34:55 ntpd[27891]: Listen normally on 3 ens1f0 172.21.15.40:123 2023-10-16T08:34:55.036 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:34:55 ntpd[27891]: Listen normally on 4 lo [::1]:123 2023-10-16T08:34:55.036 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:34:55 ntpd[27891]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:12cc%4]:123 2023-10-16T08:34:55.036 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:34:55 ntpd[27891]: Listening on routing socket on fd #22 for interface updates 2023-10-16T08:34:55.051 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:34:55 ntpd[27968]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-10-16T08:34:55.052 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:34:55 ntpd[27968]: Command line: ntpd -gq 2023-10-16T08:34:55.052 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:34:55 ntpd[27968]: ---------------------------------------------------- 2023-10-16T08:34:55.052 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:34:55 ntpd[27968]: ntp-4 is maintained by Network Time Foundation, 2023-10-16T08:34:55.052 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:34:55 ntpd[27968]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-10-16T08:34:55.053 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:34:55 ntpd[27968]: corporation. Support and training for ntp-4 are 2023-10-16T08:34:55.053 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:34:55 ntpd[27968]: available at https://www.nwtime.org/support 2023-10-16T08:34:55.053 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:34:55 ntpd[27968]: ---------------------------------------------------- 2023-10-16T08:34:55.054 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:34:55 ntpd[27968]: proto: precision = 0.063 usec (-24) 2023-10-16T08:34:55.054 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:34:55 ntpd[27968]: basedate set to 2022-02-04 2023-10-16T08:34:55.054 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:34:55 ntpd[27968]: gps base set to 2022-02-06 (week 2196) 2023-10-16T08:34:55.055 INFO:teuthology.orchestra.run.smithi086.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-16T08:34:55.055 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:34:55 ntpd[27968]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-10-16T08:34:55.055 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:34:55 ntpd[27968]: restrict ::: KOD does nothing without LIMITED. 2023-10-16T08:34:55.056 INFO:teuthology.orchestra.run.smithi086.stderr:restrict ::: KOD does nothing without LIMITED. 2023-10-16T08:34:55.057 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:34:55 ntpd[27968]: Listen and drop on 0 v6wildcard [::]:123 2023-10-16T08:34:55.057 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:34:55 ntpd[27968]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-10-16T08:34:55.057 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:34:55 ntpd[27968]: Listen normally on 2 lo 127.0.0.1:123 2023-10-16T08:34:55.057 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:34:55 ntpd[27968]: Listen normally on 3 enp3s0f1 172.21.15.86:123 2023-10-16T08:34:55.058 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:34:55 ntpd[27968]: Listen normally on 4 lo [::1]:123 2023-10-16T08:34:55.058 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:34:55 ntpd[27968]: Listen normally on 5 enp3s0f1 [fe80::ec4:7aff:fe8f:d809%5]:123 2023-10-16T08:34:55.058 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:34:55 ntpd[27968]: Listening on routing socket on fd #22 for interface updates 2023-10-16T08:34:56.052 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:56 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:34:56.052 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:56 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:34:56.053 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:56 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:34:56.053 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:56 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:34:56.053 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:56 ntpd[27968]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:34:56.053 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:56 ntpd[27968]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:34:58.030 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:34:58 ntpd[27891]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:34:58.030 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:34:58 ntpd[27891]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:34:58.030 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:34:58 ntpd[27891]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:34:58.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:34:58 ntpd[27891]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:34:58.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:34:58 ntpd[27891]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:34:58.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:34:58 ntpd[27891]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:34:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:58 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:34:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:58 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:34:58.053 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:58 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:34:58.054 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:58 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:34:58.054 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:58 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:34:58.054 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:58 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:34:58.054 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:58 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:34:58.054 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:58 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:34:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:58 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:34:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:58 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:34:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:58 ntpd[27968]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:34:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:58 ntpd[27968]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:34:58.055 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:58 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:34:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:58 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:34:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:58 ntpd[27968]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:34:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:58 ntpd[27968]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:34:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:58 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:34:58.056 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:58 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:34:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:58 ntpd[27968]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:34:58.057 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:34:58 ntpd[27968]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:34:59.030 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:34:59 ntpd[27891]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:34:59.030 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:34:59 ntpd[27891]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:34:59.030 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:34:59 ntpd[27891]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:34:59.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:34:59 ntpd[27891]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:34:59.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:34:59 ntpd[27891]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:34:59.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:34:59 ntpd[27891]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:35:00.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:00 ntpd[27891]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:00.032 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:00 ntpd[27891]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:00.032 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:00 ntpd[27891]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:00.032 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:00 ntpd[27891]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:00.032 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:00 ntpd[27891]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:00.032 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:00 ntpd[27891]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:00.033 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:00 ntpd[27891]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:00.033 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:00 ntpd[27891]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:00.033 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:00 ntpd[27891]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:35:00.033 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:00 ntpd[27891]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:35:00.033 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:00 ntpd[27891]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:00.033 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:00 ntpd[27891]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:00.034 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:00 ntpd[27891]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:35:00.034 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:00 ntpd[27891]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:35:00.053 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:00 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:00.054 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:00 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:00.054 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:00 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:00.054 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:00 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:00.054 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:00 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:00.055 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:00 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:00.055 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:00 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:00.055 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:00 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:00.055 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:00 ntpd[27968]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:35:00.055 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:00 ntpd[27968]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:35:00.056 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:00 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:00.056 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:00 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:00.056 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:00 ntpd[27968]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:35:00.056 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:00 ntpd[27968]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:35:00.056 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:00 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:00.057 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:00 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:00.057 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:00 ntpd[27968]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:35:00.057 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:00 ntpd[27968]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:35:01.030 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:01 ntpd[27891]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:01.030 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:01 ntpd[27891]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:01.030 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:01 ntpd[27891]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:01.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:01 ntpd[27891]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:01.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:01 ntpd[27891]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:35:01.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:01 ntpd[27891]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:35:02.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:02 ntpd[27891]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:02.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:02 ntpd[27891]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:02.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:02 ntpd[27891]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:02.032 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:02 ntpd[27891]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:02.032 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:02 ntpd[27891]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:02.032 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:02 ntpd[27891]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:02.032 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:02 ntpd[27891]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:35:02.033 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:02 ntpd[27891]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:35:02.033 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:02 ntpd[27891]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:02.033 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:02 ntpd[27891]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:02.033 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:02 ntpd[27891]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:35:02.033 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:02 ntpd[27891]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:35:02.052 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:02 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:02.053 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:02 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:02.053 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:02 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:02.053 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:02 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:02.054 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:02 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:02.054 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:02 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:02.054 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:02 ntpd[27968]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:02.054 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:02 ntpd[27968]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:02.055 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:02 ntpd[27968]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:35:02.055 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:02 ntpd[27968]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:35:02.055 INFO:teuthology.orchestra.run.smithi086.stdout:16 Oct 08:35:02 ntpd[27968]: ntpd: time slew -0.000135 s 2023-10-16T08:35:02.056 INFO:teuthology.orchestra.run.smithi086.stdout:ntpd: time slew -0.000135s 2023-10-16T08:35:02.056 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:02 ntpd[27968]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-16T08:35:02.057 INFO:teuthology.orchestra.run.smithi086.stderr:16 Oct 08:35:02 ntpd[27968]: can't open /var/log/ntpstats/loopstats.20231016: Permission denied 2023-10-16T08:35:02.124 INFO:teuthology.orchestra.run.smithi086.stdout: remote refid st t when poll reach delay offset jitter 2023-10-16T08:35:02.125 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================== 2023-10-16T08:35:02.125 INFO:teuthology.orchestra.run.smithi086.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T08:35:02.125 INFO:teuthology.orchestra.run.smithi086.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T08:35:02.125 INFO:teuthology.orchestra.run.smithi086.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T08:35:02.126 INFO:teuthology.orchestra.run.smithi086.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T08:35:03.030 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:03 ntpd[27891]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:03.030 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:03 ntpd[27891]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:03.030 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:03 ntpd[27891]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:03.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:03 ntpd[27891]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:03.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:03 ntpd[27891]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:35:03.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:03 ntpd[27891]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:35:04.030 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:04 ntpd[27891]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:04.030 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:04 ntpd[27891]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:04.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:04 ntpd[27891]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:04.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:04 ntpd[27891]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:04.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:04 ntpd[27891]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-10-16T08:35:04.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:04 ntpd[27891]: can't open /var/log/ntpstats/rawstats.20231016: Permission denied 2023-10-16T08:35:04.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:04 ntpd[27891]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-10-16T08:35:04.031 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:04 ntpd[27891]: can't open /var/log/ntpstats/peerstats.20231016: Permission denied 2023-10-16T08:35:04.032 INFO:teuthology.orchestra.run.smithi040.stdout:16 Oct 08:35:04 ntpd[27891]: ntpd: time slew -0.000894 s 2023-10-16T08:35:04.032 INFO:teuthology.orchestra.run.smithi040.stdout:ntpd: time slew -0.000894s 2023-10-16T08:35:04.033 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:04 ntpd[27891]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-10-16T08:35:04.033 INFO:teuthology.orchestra.run.smithi040.stderr:16 Oct 08:35:04 ntpd[27891]: can't open /var/log/ntpstats/loopstats.20231016: Permission denied 2023-10-16T08:35:04.100 INFO:teuthology.orchestra.run.smithi040.stdout: remote refid st t when poll reach delay offset jitter 2023-10-16T08:35:04.100 INFO:teuthology.orchestra.run.smithi040.stdout:============================================================================== 2023-10-16T08:35:04.100 INFO:teuthology.orchestra.run.smithi040.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T08:35:04.100 INFO:teuthology.orchestra.run.smithi040.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T08:35:04.101 INFO:teuthology.orchestra.run.smithi040.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T08:35:04.101 INFO:teuthology.orchestra.run.smithi040.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T08:35:04.102 INFO:teuthology.run_tasks:Running task install... 2023-10-16T08:35:04.117 DEBUG:teuthology.task.install:project ceph 2023-10-16T08:35:04.118 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'c7471068426370a3754c2250e7729280233d552b'}} 2023-10-16T08:35:04.118 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c7471068426370a3754c2250e7729280233d552b'} 2023-10-16T08:35:04.118 INFO:teuthology.task.install:Using flavor: default 2023-10-16T08:35:04.124 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-16T08:35:04.125 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-16T08:35:04.125 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-key list | grep Ceph 2023-10-16T08:35:04.126 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-key list | grep Ceph 2023-10-16T08:35:04.191 INFO:teuthology.orchestra.run.smithi086.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-10-16T08:35:04.206 INFO:teuthology.orchestra.run.smithi040.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-10-16T08:35:04.221 INFO:teuthology.orchestra.run.smithi086.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-16T08:35:04.221 INFO:teuthology.orchestra.run.smithi086.stdout:uid [ unknown] Ceph.com (release key) 2023-10-16T08:35:04.222 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-16T08:35:04.222 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-10-16T08:35:04.223 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c7471068426370a3754c2250e7729280233d552b 2023-10-16T08:35:04.238 INFO:teuthology.orchestra.run.smithi040.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-10-16T08:35:04.238 INFO:teuthology.orchestra.run.smithi040.stdout:uid [ unknown] Ceph.com (release key) 2023-10-16T08:35:04.239 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-10-16T08:35:04.239 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-10-16T08:35:04.239 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c7471068426370a3754c2250e7729280233d552b 2023-10-16T08:35:04.444 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default/ 2023-10-16T08:35:04.444 INFO:teuthology.task.install.deb:Package version is 18.0.0-6675-gc7471068-1jammy 2023-10-16T08:35:04.451 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default/ 2023-10-16T08:35:04.452 INFO:teuthology.task.install.deb:Package version is 18.0.0-6675-gc7471068-1jammy 2023-10-16T08:35:04.554 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:35:04.555 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-16T08:35:04.569 DEBUG:teuthology.orchestra.run.smithi086:> sudo apt-get update 2023-10-16T08:35:04.617 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:35:04.618 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-10-16T08:35:04.632 DEBUG:teuthology.orchestra.run.smithi040:> sudo apt-get update 2023-10-16T08:35:04.733 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-16T08:35:04.744 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-10-16T08:35:04.749 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:3 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-16T08:35:04.761 INFO:teuthology.orchestra.run.smithi086.stdout:Hit:4 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-10-16T08:35:04.803 INFO:teuthology.orchestra.run.smithi086.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy InRelease 2023-10-16T08:35:04.805 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:1 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-10-16T08:35:04.825 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-10-16T08:35:04.848 INFO:teuthology.orchestra.run.smithi086.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy Release.gpg 2023-10-16T08:35:04.870 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,415 B] 2023-10-16T08:35:04.901 INFO:teuthology.orchestra.run.smithi086.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-10-16T08:35:04.920 INFO:teuthology.orchestra.run.smithi040.stdout:Ign:2 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy InRelease 2023-10-16T08:35:04.925 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-10-16T08:35:04.958 INFO:teuthology.orchestra.run.smithi040.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-10-16T08:35:04.996 INFO:teuthology.orchestra.run.smithi040.stdout:Ign:5 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy Release.gpg 2023-10-16T08:35:05.006 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:6 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-10-16T08:35:05.032 INFO:teuthology.orchestra.run.smithi040.stdout:Get:7 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.5 kB] 2023-10-16T08:35:05.070 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 28.6 kB in 0s (87.7 kB/s) 2023-10-16T08:35:05.090 INFO:teuthology.orchestra.run.smithi040.stdout:Hit:8 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-10-16T08:35:05.111 INFO:teuthology.orchestra.run.smithi040.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,415 B] 2023-10-16T08:35:05.234 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 28.6 kB in 0s (68.7 kB/s) 2023-10-16T08:35:05.909 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2023-10-16T08:35:05.925 DEBUG:teuthology.orchestra.run.smithi086:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-6675-gc7471068-1jammy cephadm=18.0.0-6675-gc7471068-1jammy ceph-mds=18.0.0-6675-gc7471068-1jammy ceph-mgr=18.0.0-6675-gc7471068-1jammy ceph-common=18.0.0-6675-gc7471068-1jammy ceph-fuse=18.0.0-6675-gc7471068-1jammy ceph-test=18.0.0-6675-gc7471068-1jammy ceph-volume=18.0.0-6675-gc7471068-1jammy radosgw=18.0.0-6675-gc7471068-1jammy python3-rados=18.0.0-6675-gc7471068-1jammy python3-rgw=18.0.0-6675-gc7471068-1jammy python3-cephfs=18.0.0-6675-gc7471068-1jammy python3-rbd=18.0.0-6675-gc7471068-1jammy libcephfs2=18.0.0-6675-gc7471068-1jammy libcephfs-dev=18.0.0-6675-gc7471068-1jammy librados2=18.0.0-6675-gc7471068-1jammy librbd1=18.0.0-6675-gc7471068-1jammy rbd-fuse=18.0.0-6675-gc7471068-1jammy python3-cephfs=18.0.0-6675-gc7471068-1jammy cephfs-shell=18.0.0-6675-gc7471068-1jammy cephfs-top=18.0.0-6675-gc7471068-1jammy cephfs-mirror=18.0.0-6675-gc7471068-1jammy 2023-10-16T08:35:05.984 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2023-10-16T08:35:06.136 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2023-10-16T08:35:06.150 DEBUG:teuthology.orchestra.run.smithi040:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.0.0-6675-gc7471068-1jammy cephadm=18.0.0-6675-gc7471068-1jammy ceph-mds=18.0.0-6675-gc7471068-1jammy ceph-mgr=18.0.0-6675-gc7471068-1jammy ceph-common=18.0.0-6675-gc7471068-1jammy ceph-fuse=18.0.0-6675-gc7471068-1jammy ceph-test=18.0.0-6675-gc7471068-1jammy ceph-volume=18.0.0-6675-gc7471068-1jammy radosgw=18.0.0-6675-gc7471068-1jammy python3-rados=18.0.0-6675-gc7471068-1jammy python3-rgw=18.0.0-6675-gc7471068-1jammy python3-cephfs=18.0.0-6675-gc7471068-1jammy python3-rbd=18.0.0-6675-gc7471068-1jammy libcephfs2=18.0.0-6675-gc7471068-1jammy libcephfs-dev=18.0.0-6675-gc7471068-1jammy librados2=18.0.0-6675-gc7471068-1jammy librbd1=18.0.0-6675-gc7471068-1jammy rbd-fuse=18.0.0-6675-gc7471068-1jammy python3-cephfs=18.0.0-6675-gc7471068-1jammy cephfs-shell=18.0.0-6675-gc7471068-1jammy cephfs-top=18.0.0-6675-gc7471068-1jammy cephfs-mirror=18.0.0-6675-gc7471068-1jammy 2023-10-16T08:35:06.164 INFO:teuthology.orchestra.run.smithi086.stdout:Building dependency tree... 2023-10-16T08:35:06.165 INFO:teuthology.orchestra.run.smithi086.stdout:Reading state information... 2023-10-16T08:35:06.211 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2023-10-16T08:35:06.312 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages were automatically installed and are no longer required: 2023-10-16T08:35:06.313 INFO:teuthology.orchestra.run.smithi086.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-16T08:35:06.313 INFO:teuthology.orchestra.run.smithi086.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-16T08:35:06.313 INFO:teuthology.orchestra.run.smithi086.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-16T08:35:06.313 INFO:teuthology.orchestra.run.smithi086.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-16T08:35:06.314 INFO:teuthology.orchestra.run.smithi086.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-16T08:35:06.314 INFO:teuthology.orchestra.run.smithi086.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-16T08:35:06.315 INFO:teuthology.orchestra.run.smithi086.stdout:The following additional packages will be installed: 2023-10-16T08:35:06.315 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-16T08:35:06.315 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-10-16T08:35:06.316 INFO:teuthology.orchestra.run.smithi086.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-16T08:35:06.316 INFO:teuthology.orchestra.run.smithi086.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-10-16T08:35:06.316 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-10-16T08:35:06.316 INFO:teuthology.orchestra.run.smithi086.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-10-16T08:35:06.317 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-10-16T08:35:06.317 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-10-16T08:35:06.317 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-10-16T08:35:06.317 INFO:teuthology.orchestra.run.smithi086.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-10-16T08:35:06.317 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-10-16T08:35:06.317 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-10-16T08:35:06.318 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-10-16T08:35:06.318 INFO:teuthology.orchestra.run.smithi086.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-10-16T08:35:06.318 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-10-16T08:35:06.318 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-10-16T08:35:06.319 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-10-16T08:35:06.319 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-10-16T08:35:06.319 INFO:teuthology.orchestra.run.smithi086.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-10-16T08:35:06.319 INFO:teuthology.orchestra.run.smithi086.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-10-16T08:35:06.319 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-10-16T08:35:06.320 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-10-16T08:35:06.320 INFO:teuthology.orchestra.run.smithi086.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-16T08:35:06.320 INFO:teuthology.orchestra.run.smithi086.stdout:Suggested packages: 2023-10-16T08:35:06.320 INFO:teuthology.orchestra.run.smithi086.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-10-16T08:35:06.320 INFO:teuthology.orchestra.run.smithi086.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-10-16T08:35:06.321 INFO:teuthology.orchestra.run.smithi086.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-10-16T08:35:06.321 INFO:teuthology.orchestra.run.smithi086.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-10-16T08:35:06.321 INFO:teuthology.orchestra.run.smithi086.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-10-16T08:35:06.321 INFO:teuthology.orchestra.run.smithi086.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-10-16T08:35:06.321 INFO:teuthology.orchestra.run.smithi086.stdout: python3-watchdog 2023-10-16T08:35:06.322 INFO:teuthology.orchestra.run.smithi086.stdout:Recommended packages: 2023-10-16T08:35:06.322 INFO:teuthology.orchestra.run.smithi086.stdout: btrfs-tools 2023-10-16T08:35:06.400 INFO:teuthology.orchestra.run.smithi040.stdout:Building dependency tree... 2023-10-16T08:35:06.401 INFO:teuthology.orchestra.run.smithi040.stdout:Reading state information... 2023-10-16T08:35:06.415 INFO:teuthology.orchestra.run.smithi086.stdout:The following NEW packages will be installed: 2023-10-16T08:35:06.415 INFO:teuthology.orchestra.run.smithi086.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-16T08:35:06.415 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-16T08:35:06.416 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-16T08:35:06.416 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-10-16T08:35:06.417 INFO:teuthology.orchestra.run.smithi086.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-16T08:35:06.417 INFO:teuthology.orchestra.run.smithi086.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-10-16T08:35:06.417 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-10-16T08:35:06.418 INFO:teuthology.orchestra.run.smithi086.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-10-16T08:35:06.418 INFO:teuthology.orchestra.run.smithi086.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-10-16T08:35:06.418 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-10-16T08:35:06.418 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-10-16T08:35:06.418 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-10-16T08:35:06.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-10-16T08:35:06.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-10-16T08:35:06.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-10-16T08:35:06.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-10-16T08:35:06.419 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-10-16T08:35:06.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-10-16T08:35:06.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-10-16T08:35:06.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-10-16T08:35:06.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-16T08:35:06.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-10-16T08:35:06.420 INFO:teuthology.orchestra.run.smithi086.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-10-16T08:35:06.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-10-16T08:35:06.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-10-16T08:35:06.421 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-10-16T08:35:06.421 INFO:teuthology.orchestra.run.smithi086.stdout: socat xmlstarlet 2023-10-16T08:35:06.577 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages were automatically installed and are no longer required: 2023-10-16T08:35:06.577 INFO:teuthology.orchestra.run.smithi040.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-16T08:35:06.577 INFO:teuthology.orchestra.run.smithi040.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-16T08:35:06.578 INFO:teuthology.orchestra.run.smithi040.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-16T08:35:06.579 INFO:teuthology.orchestra.run.smithi040.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-16T08:35:06.579 INFO:teuthology.orchestra.run.smithi040.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-16T08:35:06.579 INFO:teuthology.orchestra.run.smithi040.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-16T08:35:06.579 INFO:teuthology.orchestra.run.smithi040.stdout:The following additional packages will be installed: 2023-10-16T08:35:06.579 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-10-16T08:35:06.580 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-10-16T08:35:06.580 INFO:teuthology.orchestra.run.smithi040.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-16T08:35:06.580 INFO:teuthology.orchestra.run.smithi040.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-10-16T08:35:06.581 INFO:teuthology.orchestra.run.smithi040.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-10-16T08:35:06.581 INFO:teuthology.orchestra.run.smithi040.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-10-16T08:35:06.581 INFO:teuthology.orchestra.run.smithi040.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-10-16T08:35:06.582 INFO:teuthology.orchestra.run.smithi040.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-10-16T08:35:06.582 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-10-16T08:35:06.582 INFO:teuthology.orchestra.run.smithi040.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-10-16T08:35:06.582 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-10-16T08:35:06.582 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-10-16T08:35:06.583 INFO:teuthology.orchestra.run.smithi040.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-10-16T08:35:06.583 INFO:teuthology.orchestra.run.smithi040.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-10-16T08:35:06.583 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-10-16T08:35:06.583 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-10-16T08:35:06.583 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-10-16T08:35:06.584 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-10-16T08:35:06.584 INFO:teuthology.orchestra.run.smithi040.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-10-16T08:35:06.584 INFO:teuthology.orchestra.run.smithi040.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-10-16T08:35:06.584 INFO:teuthology.orchestra.run.smithi040.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-10-16T08:35:06.584 INFO:teuthology.orchestra.run.smithi040.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-10-16T08:35:06.584 INFO:teuthology.orchestra.run.smithi040.stdout: qttranslations5-l10n socat xmlstarlet 2023-10-16T08:35:06.585 INFO:teuthology.orchestra.run.smithi040.stdout:Suggested packages: 2023-10-16T08:35:06.585 INFO:teuthology.orchestra.run.smithi040.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-10-16T08:35:06.585 INFO:teuthology.orchestra.run.smithi040.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-10-16T08:35:06.585 INFO:teuthology.orchestra.run.smithi040.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-10-16T08:35:06.585 INFO:teuthology.orchestra.run.smithi040.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-10-16T08:35:06.586 INFO:teuthology.orchestra.run.smithi040.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-10-16T08:35:06.586 INFO:teuthology.orchestra.run.smithi040.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-10-16T08:35:06.586 INFO:teuthology.orchestra.run.smithi040.stdout: python3-watchdog 2023-10-16T08:35:06.586 INFO:teuthology.orchestra.run.smithi040.stdout:Recommended packages: 2023-10-16T08:35:06.586 INFO:teuthology.orchestra.run.smithi040.stdout: btrfs-tools 2023-10-16T08:35:06.602 INFO:teuthology.orchestra.run.smithi086.stdout:0 upgraded, 115 newly installed, 0 to remove and 256 not upgraded. 2023-10-16T08:35:06.602 INFO:teuthology.orchestra.run.smithi086.stdout:Need to get 184 MB of archives. 2023-10-16T08:35:06.602 INFO:teuthology.orchestra.run.smithi086.stdout:After this operation, 826 MB of additional disk space will be used. 2023-10-16T08:35:06.603 INFO:teuthology.orchestra.run.smithi086.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-10-16T08:35:06.685 INFO:teuthology.orchestra.run.smithi086.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-10-16T08:35:06.691 INFO:teuthology.orchestra.run.smithi040.stdout:The following NEW packages will be installed: 2023-10-16T08:35:06.692 INFO:teuthology.orchestra.run.smithi040.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-10-16T08:35:06.692 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-10-16T08:35:06.694 INFO:teuthology.orchestra.run.smithi040.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-10-16T08:35:06.695 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-6675-gc7471068-1jammy [3,449 kB] 2023-10-16T08:35:06.696 INFO:teuthology.orchestra.run.smithi040.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-10-16T08:35:06.696 INFO:teuthology.orchestra.run.smithi040.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-10-16T08:35:06.696 INFO:teuthology.orchestra.run.smithi040.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-10-16T08:35:06.696 INFO:teuthology.orchestra.run.smithi040.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-10-16T08:35:06.697 INFO:teuthology.orchestra.run.smithi040.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-10-16T08:35:06.697 INFO:teuthology.orchestra.run.smithi040.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-10-16T08:35:06.697 INFO:teuthology.orchestra.run.smithi040.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-10-16T08:35:06.697 INFO:teuthology.orchestra.run.smithi040.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-10-16T08:35:06.697 INFO:teuthology.orchestra.run.smithi040.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-10-16T08:35:06.698 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-10-16T08:35:06.698 INFO:teuthology.orchestra.run.smithi040.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-10-16T08:35:06.698 INFO:teuthology.orchestra.run.smithi040.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-10-16T08:35:06.698 INFO:teuthology.orchestra.run.smithi040.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-10-16T08:35:06.698 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-10-16T08:35:06.699 INFO:teuthology.orchestra.run.smithi040.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-10-16T08:35:06.699 INFO:teuthology.orchestra.run.smithi040.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-10-16T08:35:06.699 INFO:teuthology.orchestra.run.smithi040.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-10-16T08:35:06.699 INFO:teuthology.orchestra.run.smithi040.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-10-16T08:35:06.699 INFO:teuthology.orchestra.run.smithi040.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-10-16T08:35:06.699 INFO:teuthology.orchestra.run.smithi040.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-10-16T08:35:06.700 INFO:teuthology.orchestra.run.smithi040.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-10-16T08:35:06.700 INFO:teuthology.orchestra.run.smithi040.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-10-16T08:35:06.700 INFO:teuthology.orchestra.run.smithi040.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-10-16T08:35:06.700 INFO:teuthology.orchestra.run.smithi040.stdout: socat xmlstarlet 2023-10-16T08:35:06.730 INFO:teuthology.orchestra.run.smithi086.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-10-16T08:35:06.752 INFO:teuthology.orchestra.run.smithi040.stdout:0 upgraded, 115 newly installed, 0 to remove and 256 not upgraded. 2023-10-16T08:35:06.753 INFO:teuthology.orchestra.run.smithi040.stdout:Need to get 184 MB of archives. 2023-10-16T08:35:06.753 INFO:teuthology.orchestra.run.smithi040.stdout:After this operation, 826 MB of additional disk space will be used. 2023-10-16T08:35:06.753 INFO:teuthology.orchestra.run.smithi040.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-10-16T08:35:06.770 INFO:teuthology.orchestra.run.smithi040.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-10-16T08:35:06.779 INFO:teuthology.orchestra.run.smithi040.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-10-16T08:35:06.819 INFO:teuthology.orchestra.run.smithi040.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-10-16T08:35:06.829 INFO:teuthology.orchestra.run.smithi040.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-10-16T08:35:06.845 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-10-16T08:35:06.848 INFO:teuthology.orchestra.run.smithi040.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-10-16T08:35:06.858 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-10-16T08:35:06.904 INFO:teuthology.orchestra.run.smithi040.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-10-16T08:35:06.907 INFO:teuthology.orchestra.run.smithi040.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-10-16T08:35:06.919 INFO:teuthology.orchestra.run.smithi040.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-10-16T08:35:06.923 INFO:teuthology.orchestra.run.smithi040.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-10-16T08:35:06.923 INFO:teuthology.orchestra.run.smithi040.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-10-16T08:35:06.927 INFO:teuthology.orchestra.run.smithi086.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-10-16T08:35:06.932 INFO:teuthology.orchestra.run.smithi040.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-10-16T08:35:06.932 INFO:teuthology.orchestra.run.smithi040.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-10-16T08:35:06.933 INFO:teuthology.orchestra.run.smithi040.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-10-16T08:35:06.934 INFO:teuthology.orchestra.run.smithi040.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-10-16T08:35:06.934 INFO:teuthology.orchestra.run.smithi040.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-10-16T08:35:06.944 INFO:teuthology.orchestra.run.smithi040.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-10-16T08:35:06.945 INFO:teuthology.orchestra.run.smithi040.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-10-16T08:35:06.945 INFO:teuthology.orchestra.run.smithi040.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-10-16T08:35:06.961 INFO:teuthology.orchestra.run.smithi040.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-10-16T08:35:06.961 INFO:teuthology.orchestra.run.smithi040.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-10-16T08:35:06.964 INFO:teuthology.orchestra.run.smithi040.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-10-16T08:35:06.964 INFO:teuthology.orchestra.run.smithi040.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-10-16T08:35:06.965 INFO:teuthology.orchestra.run.smithi040.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-10-16T08:35:06.965 INFO:teuthology.orchestra.run.smithi040.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-10-16T08:35:06.966 INFO:teuthology.orchestra.run.smithi040.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-16T08:35:06.968 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-10-16T08:35:06.978 INFO:teuthology.orchestra.run.smithi040.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-10-16T08:35:06.978 INFO:teuthology.orchestra.run.smithi040.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-10-16T08:35:06.979 INFO:teuthology.orchestra.run.smithi040.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-10-16T08:35:06.995 INFO:teuthology.orchestra.run.smithi040.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-10-16T08:35:06.995 INFO:teuthology.orchestra.run.smithi040.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-10-16T08:35:07.001 INFO:teuthology.orchestra.run.smithi040.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-10-16T08:35:07.001 INFO:teuthology.orchestra.run.smithi040.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-10-16T08:35:07.002 INFO:teuthology.orchestra.run.smithi040.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-10-16T08:35:07.002 INFO:teuthology.orchestra.run.smithi040.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-10-16T08:35:07.003 INFO:teuthology.orchestra.run.smithi040.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-10-16T08:35:07.011 INFO:teuthology.orchestra.run.smithi040.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-10-16T08:35:07.012 INFO:teuthology.orchestra.run.smithi040.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-10-16T08:35:07.013 INFO:teuthology.orchestra.run.smithi040.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-10-16T08:35:07.023 INFO:teuthology.orchestra.run.smithi040.stdout:Get:42 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.0.0-6675-gc7471068-1jammy [3,449 kB] 2023-10-16T08:35:07.028 INFO:teuthology.orchestra.run.smithi040.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-10-16T08:35:07.030 INFO:teuthology.orchestra.run.smithi040.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-10-16T08:35:07.031 INFO:teuthology.orchestra.run.smithi040.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-10-16T08:35:07.034 INFO:teuthology.orchestra.run.smithi040.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-10-16T08:35:07.044 INFO:teuthology.orchestra.run.smithi086.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-10-16T08:35:07.055 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-10-16T08:35:07.087 INFO:teuthology.orchestra.run.smithi040.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-10-16T08:35:07.088 INFO:teuthology.orchestra.run.smithi040.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-10-16T08:35:07.089 INFO:teuthology.orchestra.run.smithi040.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-10-16T08:35:07.090 INFO:teuthology.orchestra.run.smithi040.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-16T08:35:07.090 INFO:teuthology.orchestra.run.smithi040.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-10-16T08:35:07.091 INFO:teuthology.orchestra.run.smithi040.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-10-16T08:35:07.101 INFO:teuthology.orchestra.run.smithi086.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-10-16T08:35:07.109 INFO:teuthology.orchestra.run.smithi040.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-10-16T08:35:07.111 INFO:teuthology.orchestra.run.smithi040.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-10-16T08:35:07.111 INFO:teuthology.orchestra.run.smithi040.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-10-16T08:35:07.125 INFO:teuthology.orchestra.run.smithi040.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-10-16T08:35:07.126 INFO:teuthology.orchestra.run.smithi040.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-10-16T08:35:07.126 INFO:teuthology.orchestra.run.smithi040.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-10-16T08:35:07.127 INFO:teuthology.orchestra.run.smithi040.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-10-16T08:35:07.127 INFO:teuthology.orchestra.run.smithi040.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-10-16T08:35:07.128 INFO:teuthology.orchestra.run.smithi086.stdout:Get:10 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-6675-gc7471068-1jammy [3,130 kB] 2023-10-16T08:35:07.128 INFO:teuthology.orchestra.run.smithi040.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-10-16T08:35:07.140 INFO:teuthology.orchestra.run.smithi040.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-10-16T08:35:07.140 INFO:teuthology.orchestra.run.smithi040.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-10-16T08:35:07.143 INFO:teuthology.orchestra.run.smithi040.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-10-16T08:35:07.159 INFO:teuthology.orchestra.run.smithi040.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-10-16T08:35:07.160 INFO:teuthology.orchestra.run.smithi040.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-10-16T08:35:07.161 INFO:teuthology.orchestra.run.smithi040.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-10-16T08:35:07.164 INFO:teuthology.orchestra.run.smithi040.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-10-16T08:35:07.166 INFO:teuthology.orchestra.run.smithi040.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-10-16T08:35:07.166 INFO:teuthology.orchestra.run.smithi040.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-10-16T08:35:07.170 INFO:teuthology.orchestra.run.smithi040.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-10-16T08:35:07.176 INFO:teuthology.orchestra.run.smithi040.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-10-16T08:35:07.176 INFO:teuthology.orchestra.run.smithi040.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-16T08:35:07.177 INFO:teuthology.orchestra.run.smithi040.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-10-16T08:35:07.195 INFO:teuthology.orchestra.run.smithi040.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-10-16T08:35:07.197 INFO:teuthology.orchestra.run.smithi040.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-10-16T08:35:07.197 INFO:teuthology.orchestra.run.smithi040.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-10-16T08:35:07.203 INFO:teuthology.orchestra.run.smithi040.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-10-16T08:35:07.203 INFO:teuthology.orchestra.run.smithi040.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-10-16T08:35:07.204 INFO:teuthology.orchestra.run.smithi040.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-10-16T08:35:07.205 INFO:teuthology.orchestra.run.smithi040.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-10-16T08:35:07.212 INFO:teuthology.orchestra.run.smithi040.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-10-16T08:35:07.269 INFO:teuthology.orchestra.run.smithi086.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-10-16T08:35:07.277 INFO:teuthology.orchestra.run.smithi086.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-10-16T08:35:07.285 INFO:teuthology.orchestra.run.smithi086.stdout:Get:13 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-6675-gc7471068-1jammy [688 kB] 2023-10-16T08:35:07.304 INFO:teuthology.orchestra.run.smithi086.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-10-16T08:35:07.319 INFO:teuthology.orchestra.run.smithi086.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-6675-gc7471068-1jammy [356 kB] 2023-10-16T08:35:07.326 INFO:teuthology.orchestra.run.smithi086.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-6675-gc7471068-1jammy [32.8 kB] 2023-10-16T08:35:07.327 INFO:teuthology.orchestra.run.smithi086.stdout:Get:17 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-6675-gc7471068-1jammy [180 kB] 2023-10-16T08:35:07.339 INFO:teuthology.orchestra.run.smithi086.stdout:Get:18 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-6675-gc7471068-1jammy [62.4 kB] 2023-10-16T08:35:07.384 INFO:teuthology.orchestra.run.smithi086.stdout:Get:19 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-6675-gc7471068-1jammy [329 kB] 2023-10-16T08:35:07.477 INFO:teuthology.orchestra.run.smithi086.stdout:Get:20 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-6675-gc7471068-1jammy [6,226 kB] 2023-10-16T08:35:07.478 INFO:teuthology.orchestra.run.smithi086.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-10-16T08:35:07.674 INFO:teuthology.orchestra.run.smithi086.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-10-16T08:35:07.739 INFO:teuthology.orchestra.run.smithi086.stdout:Get:23 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-6675-gc7471068-1jammy [112 kB] 2023-10-16T08:35:07.742 INFO:teuthology.orchestra.run.smithi086.stdout:Get:24 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-6675-gc7471068-1jammy [439 kB] 2023-10-16T08:35:07.829 INFO:teuthology.orchestra.run.smithi086.stdout:Get:25 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-6675-gc7471068-1jammy [22.6 MB] 2023-10-16T08:35:08.042 INFO:teuthology.orchestra.run.smithi086.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-10-16T08:35:08.049 INFO:teuthology.orchestra.run.smithi086.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-10-16T08:35:08.051 INFO:teuthology.orchestra.run.smithi086.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-10-16T08:35:08.063 INFO:teuthology.orchestra.run.smithi086.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-10-16T08:35:08.063 INFO:teuthology.orchestra.run.smithi086.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-10-16T08:35:08.065 INFO:teuthology.orchestra.run.smithi086.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-10-16T08:35:08.066 INFO:teuthology.orchestra.run.smithi086.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-10-16T08:35:08.069 INFO:teuthology.orchestra.run.smithi086.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-10-16T08:35:08.070 INFO:teuthology.orchestra.run.smithi086.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-10-16T08:35:08.235 INFO:teuthology.orchestra.run.smithi086.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-10-16T08:35:08.449 INFO:teuthology.orchestra.run.smithi040.stdout:Get:83 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.0.0-6675-gc7471068-1jammy [3,130 kB] 2023-10-16T08:35:08.676 INFO:teuthology.orchestra.run.smithi086.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-10-16T08:35:08.686 INFO:teuthology.orchestra.run.smithi086.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-10-16T08:35:08.691 INFO:teuthology.orchestra.run.smithi086.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-10-16T08:35:08.695 INFO:teuthology.orchestra.run.smithi086.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-10-16T08:35:08.714 INFO:teuthology.orchestra.run.smithi086.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-10-16T08:35:08.717 INFO:teuthology.orchestra.run.smithi086.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-10-16T08:35:08.719 INFO:teuthology.orchestra.run.smithi086.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-10-16T08:35:08.823 INFO:teuthology.orchestra.run.smithi086.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-10-16T08:35:08.825 INFO:teuthology.orchestra.run.smithi086.stdout:Get:44 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-6675-gc7471068-1jammy [6,851 kB] 2023-10-16T08:35:08.825 INFO:teuthology.orchestra.run.smithi086.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-10-16T08:35:08.825 INFO:teuthology.orchestra.run.smithi086.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-10-16T08:35:09.039 INFO:teuthology.orchestra.run.smithi086.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-10-16T08:35:09.084 INFO:teuthology.orchestra.run.smithi040.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.0.0-6675-gc7471068-1jammy [688 kB] 2023-10-16T08:35:09.091 INFO:teuthology.orchestra.run.smithi086.stdout:Get:48 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-6675-gc7471068-1jammy [239 kB] 2023-10-16T08:35:09.096 INFO:teuthology.orchestra.run.smithi086.stdout:Get:49 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-6675-gc7471068-1jammy [128 kB] 2023-10-16T08:35:09.099 INFO:teuthology.orchestra.run.smithi086.stdout:Get:50 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-6675-gc7471068-1jammy [1,496 kB] 2023-10-16T08:35:09.123 INFO:teuthology.orchestra.run.smithi086.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-10-16T08:35:09.124 INFO:teuthology.orchestra.run.smithi086.stdout:Get:52 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-6675-gc7471068-1jammy [9,863 kB] 2023-10-16T08:35:09.153 INFO:teuthology.orchestra.run.smithi040.stdout:Get:85 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.0.0-6675-gc7471068-1jammy [356 kB] 2023-10-16T08:35:09.213 INFO:teuthology.orchestra.run.smithi086.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-10-16T08:35:09.271 INFO:teuthology.orchestra.run.smithi040.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.0.0-6675-gc7471068-1jammy [32.8 kB] 2023-10-16T08:35:09.273 INFO:teuthology.orchestra.run.smithi040.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.0.0-6675-gc7471068-1jammy [180 kB] 2023-10-16T08:35:09.274 INFO:teuthology.orchestra.run.smithi086.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-10-16T08:35:09.280 INFO:teuthology.orchestra.run.smithi040.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.0.0-6675-gc7471068-1jammy [62.4 kB] 2023-10-16T08:35:09.282 INFO:teuthology.orchestra.run.smithi040.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.0.0-6675-gc7471068-1jammy [329 kB] 2023-10-16T08:35:09.342 INFO:teuthology.orchestra.run.smithi040.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.0.0-6675-gc7471068-1jammy [6,226 kB] 2023-10-16T08:35:09.394 INFO:teuthology.orchestra.run.smithi086.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-10-16T08:35:09.479 INFO:teuthology.orchestra.run.smithi086.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-10-16T08:35:09.499 INFO:teuthology.orchestra.run.smithi086.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-urllib3 all 1.26.5-1~exp1 [96.4 kB] 2023-10-16T08:35:09.526 INFO:teuthology.orchestra.run.smithi086.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-10-16T08:35:09.538 INFO:teuthology.orchestra.run.smithi086.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-10-16T08:35:09.557 INFO:teuthology.orchestra.run.smithi086.stdout:Get:60 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-6675-gc7471068-1jammy [31.7 MB] 2023-10-16T08:35:09.576 INFO:teuthology.orchestra.run.smithi086.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-10-16T08:35:09.585 INFO:teuthology.orchestra.run.smithi086.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-10-16T08:35:09.779 INFO:teuthology.orchestra.run.smithi086.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-10-16T08:35:10.401 INFO:teuthology.orchestra.run.smithi040.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.0.0-6675-gc7471068-1jammy [112 kB] 2023-10-16T08:35:10.404 INFO:teuthology.orchestra.run.smithi040.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.0.0-6675-gc7471068-1jammy [439 kB] 2023-10-16T08:35:10.478 INFO:teuthology.orchestra.run.smithi040.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.0.0-6675-gc7471068-1jammy [22.6 MB] 2023-10-16T08:35:10.605 INFO:teuthology.orchestra.run.smithi086.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-10-16T08:35:10.607 INFO:teuthology.orchestra.run.smithi086.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-10-16T08:35:10.609 INFO:teuthology.orchestra.run.smithi086.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-10-16T08:35:10.611 INFO:teuthology.orchestra.run.smithi086.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-10-16T08:35:10.611 INFO:teuthology.orchestra.run.smithi086.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-10-16T08:35:10.614 INFO:teuthology.orchestra.run.smithi086.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-10-16T08:35:10.657 INFO:teuthology.orchestra.run.smithi086.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-10-16T08:35:10.660 INFO:teuthology.orchestra.run.smithi086.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-10-16T08:35:10.660 INFO:teuthology.orchestra.run.smithi086.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-10-16T08:35:10.716 INFO:teuthology.orchestra.run.smithi086.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-10-16T08:35:10.880 INFO:teuthology.orchestra.run.smithi086.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-10-16T08:35:10.979 INFO:teuthology.orchestra.run.smithi086.stdout:Get:75 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-6675-gc7471068-1jammy [14.1 kB] 2023-10-16T08:35:10.980 INFO:teuthology.orchestra.run.smithi086.stdout:Get:76 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-6675-gc7471068-1jammy [839 kB] 2023-10-16T08:35:10.996 INFO:teuthology.orchestra.run.smithi086.stdout:Get:77 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-6675-gc7471068-1jammy [2,146 kB] 2023-10-16T08:35:11.034 INFO:teuthology.orchestra.run.smithi086.stdout:Get:78 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-6675-gc7471068-1jammy [283 kB] 2023-10-16T08:35:11.038 INFO:teuthology.orchestra.run.smithi086.stdout:Get:79 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-6675-gc7471068-1jammy [138 kB] 2023-10-16T08:35:11.040 INFO:teuthology.orchestra.run.smithi086.stdout:Get:80 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-6675-gc7471068-1jammy [1,906 kB] 2023-10-16T08:35:11.082 INFO:teuthology.orchestra.run.smithi086.stdout:Get:81 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-6675-gc7471068-1jammy [8,625 kB] 2023-10-16T08:35:11.139 INFO:teuthology.orchestra.run.smithi086.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-10-16T08:35:11.172 INFO:teuthology.orchestra.run.smithi086.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-10-16T08:35:11.198 INFO:teuthology.orchestra.run.smithi086.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-10-16T08:35:11.249 INFO:teuthology.orchestra.run.smithi086.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-10-16T08:35:11.270 INFO:teuthology.orchestra.run.smithi086.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-10-16T08:35:11.298 INFO:teuthology.orchestra.run.smithi086.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-10-16T08:35:11.426 INFO:teuthology.orchestra.run.smithi086.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-10-16T08:35:11.436 INFO:teuthology.orchestra.run.smithi086.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-6675-gc7471068-1jammy [14.2 kB] 2023-10-16T08:35:11.437 INFO:teuthology.orchestra.run.smithi086.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-6675-gc7471068-1jammy [47.8 MB] 2023-10-16T08:35:11.457 INFO:teuthology.orchestra.run.smithi086.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-10-16T08:35:11.477 INFO:teuthology.orchestra.run.smithi086.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-10-16T08:35:11.485 INFO:teuthology.orchestra.run.smithi086.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-10-16T08:35:11.689 INFO:teuthology.orchestra.run.smithi086.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-10-16T08:35:12.152 INFO:teuthology.orchestra.run.smithi086.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-10-16T08:35:12.157 INFO:teuthology.orchestra.run.smithi086.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-10-16T08:35:12.284 INFO:teuthology.orchestra.run.smithi086.stdout:Get:97 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-10-16T08:35:12.331 INFO:teuthology.orchestra.run.smithi086.stdout:Get:98 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-10-16T08:35:12.331 INFO:teuthology.orchestra.run.smithi086.stdout:Get:99 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-10-16T08:35:12.337 INFO:teuthology.orchestra.run.smithi086.stdout:Get:100 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-10-16T08:35:12.339 INFO:teuthology.orchestra.run.smithi086.stdout:Get:101 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-10-16T08:35:12.354 INFO:teuthology.orchestra.run.smithi086.stdout:Get:102 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-10-16T08:35:12.362 INFO:teuthology.orchestra.run.smithi086.stdout:Get:103 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-10-16T08:35:12.424 INFO:teuthology.orchestra.run.smithi086.stdout:Get:104 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-10-16T08:35:12.590 INFO:teuthology.orchestra.run.smithi086.stdout:Get:105 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-10-16T08:35:12.757 INFO:teuthology.orchestra.run.smithi086.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-10-16T08:35:13.036 INFO:teuthology.orchestra.run.smithi086.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-10-16T08:35:13.062 INFO:teuthology.orchestra.run.smithi086.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-10-16T08:35:14.109 INFO:teuthology.orchestra.run.smithi086.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-6675-gc7471068-1jammy [120 kB] 2023-10-16T08:35:14.181 INFO:teuthology.orchestra.run.smithi086.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-6675-gc7471068-1jammy [227 kB] 2023-10-16T08:35:14.186 INFO:teuthology.orchestra.run.smithi086.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-6675-gc7471068-1jammy [34.0 kB] 2023-10-16T08:35:14.187 INFO:teuthology.orchestra.run.smithi086.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-6675-gc7471068-1jammy [23.4 kB] 2023-10-16T08:35:14.187 INFO:teuthology.orchestra.run.smithi086.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-6675-gc7471068-1jammy [38.3 kB] 2023-10-16T08:35:14.188 INFO:teuthology.orchestra.run.smithi086.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-6675-gc7471068-1jammy [12.3 MB] 2023-10-16T08:35:14.560 INFO:teuthology.orchestra.run.smithi086.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-6675-gc7471068-1jammy [91.6 kB] 2023-10-16T08:35:14.728 INFO:teuthology.orchestra.run.smithi040.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.0.0-6675-gc7471068-1jammy [6,851 kB] 2023-10-16T08:35:15.117 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 184 MB in 8s (22.6 MB/s) 2023-10-16T08:35:15.206 INFO:teuthology.orchestra.run.smithi040.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.0.0-6675-gc7471068-1jammy [239 kB] 2023-10-16T08:35:15.212 INFO:teuthology.orchestra.run.smithi040.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.0.0-6675-gc7471068-1jammy [128 kB] 2023-10-16T08:35:15.239 INFO:teuthology.orchestra.run.smithi040.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.0.0-6675-gc7471068-1jammy [1,496 kB] 2023-10-16T08:35:15.250 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-16T08:35:15.343 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165988 files and directories currently installed.) 2023-10-16T08:35:15.347 INFO:teuthology.orchestra.run.smithi040.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.0.0-6675-gc7471068-1jammy [9,863 kB] 2023-10-16T08:35:15.348 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-10-16T08:35:15.390 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-10-16T08:35:15.659 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-16T08:35:15.686 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-10-16T08:35:15.726 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-16T08:35:16.029 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cmd2. 2023-10-16T08:35:16.055 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-10-16T08:35:16.058 INFO:teuthology.orchestra.run.smithi040.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.0.0-6675-gc7471068-1jammy [31.7 MB] 2023-10-16T08:35:16.095 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-10-16T08:35:16.407 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libfmt8:amd64. 2023-10-16T08:35:16.433 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-10-16T08:35:16.541 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-16T08:35:16.876 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-10-16T08:35:16.903 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-10-16T08:35:16.943 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-16T08:35:17.288 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-16T08:35:17.315 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-10-16T08:35:17.355 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-16T08:35:17.624 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-16T08:35:17.651 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-10-16T08:35:17.691 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-16T08:35:18.128 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-16T08:35:18.154 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-16T08:35:18.202 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:35:18.581 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-16T08:35:18.607 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-16T08:35:18.609 INFO:teuthology.orchestra.run.smithi040.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.0.0-6675-gc7471068-1jammy [14.1 kB] 2023-10-16T08:35:18.610 INFO:teuthology.orchestra.run.smithi040.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.0.0-6675-gc7471068-1jammy [839 kB] 2023-10-16T08:35:18.626 INFO:teuthology.orchestra.run.smithi040.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.0.0-6675-gc7471068-1jammy [2,146 kB] 2023-10-16T08:35:18.639 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:35:18.761 INFO:teuthology.orchestra.run.smithi040.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.0.0-6675-gc7471068-1jammy [283 kB] 2023-10-16T08:35:18.795 INFO:teuthology.orchestra.run.smithi040.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.0.0-6675-gc7471068-1jammy [138 kB] 2023-10-16T08:35:18.798 INFO:teuthology.orchestra.run.smithi040.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.0.0-6675-gc7471068-1jammy [1,906 kB] 2023-10-16T08:35:18.963 INFO:teuthology.orchestra.run.smithi040.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.0.0-6675-gc7471068-1jammy [8,625 kB] 2023-10-16T08:35:18.967 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-16T08:35:18.993 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-16T08:35:19.034 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:35:19.387 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-10-16T08:35:19.413 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-10-16T08:35:19.453 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-16T08:35:19.635 INFO:teuthology.orchestra.run.smithi040.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.0.0-6675-gc7471068-1jammy [14.2 kB] 2023-10-16T08:35:19.635 INFO:teuthology.orchestra.run.smithi040.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.0.0-6675-gc7471068-1jammy [47.8 MB] 2023-10-16T08:35:19.757 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librados2. 2023-10-16T08:35:19.783 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../011-librados2_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:19.823 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librados2 (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:20.259 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librbd1. 2023-10-16T08:35:20.285 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../012-librbd1_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:20.326 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librbd1 (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:20.753 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libcephfs2. 2023-10-16T08:35:20.780 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../013-libcephfs2_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:20.820 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libcephfs2 (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:21.106 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rados. 2023-10-16T08:35:21.132 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../014-python3-rados_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:21.172 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rados (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:21.459 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-16T08:35:21.485 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.0.0-6675-gc7471068-1jammy_all.deb ... 2023-10-16T08:35:21.517 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-ceph-argparse (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:21.752 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cephfs. 2023-10-16T08:35:21.778 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../016-python3-cephfs_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:21.819 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cephfs (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:22.182 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-16T08:35:22.207 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../017-python3-ceph-common_18.0.0-6675-gc7471068-1jammy_all.deb ... 2023-10-16T08:35:22.247 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-ceph-common (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:22.575 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-prettytable. 2023-10-16T08:35:22.601 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-10-16T08:35:22.642 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-10-16T08:35:22.877 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rbd. 2023-10-16T08:35:22.893 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../019-python3-rbd_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:22.915 INFO:teuthology.orchestra.run.smithi040.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.0.0-6675-gc7471068-1jammy [120 kB] 2023-10-16T08:35:22.927 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rbd (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:22.990 INFO:teuthology.orchestra.run.smithi040.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.0.0-6675-gc7471068-1jammy [227 kB] 2023-10-16T08:35:22.994 INFO:teuthology.orchestra.run.smithi040.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.0.0-6675-gc7471068-1jammy [34.0 kB] 2023-10-16T08:35:22.995 INFO:teuthology.orchestra.run.smithi040.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.0.0-6675-gc7471068-1jammy [23.4 kB] 2023-10-16T08:35:22.996 INFO:teuthology.orchestra.run.smithi040.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.0.0-6675-gc7471068-1jammy [38.3 kB] 2023-10-16T08:35:22.997 INFO:teuthology.orchestra.run.smithi040.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.0.0-6675-gc7471068-1jammy [12.3 MB] 2023-10-16T08:35:23.247 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-16T08:35:23.274 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-10-16T08:35:23.314 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-16T08:35:23.616 INFO:teuthology.orchestra.run.smithi040.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments/c7471068426370a3754c2250e7729280233d552b/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.0.0-6675-gc7471068-1jammy [91.6 kB] 2023-10-16T08:35:23.642 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package librgw2. 2023-10-16T08:35:23.668 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../021-librgw2_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:23.725 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking librgw2 (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:24.195 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 184 MB in 17s (10.9 MB/s) 2023-10-16T08:35:24.260 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rgw. 2023-10-16T08:35:24.276 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../022-python3-rgw_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:24.319 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rgw (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:24.322 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pyperclip. 2023-10-16T08:35:24.422 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 165988 files and directories currently installed.) 2023-10-16T08:35:24.428 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-10-16T08:35:24.470 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-10-16T08:35:24.605 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liboath0:amd64. 2023-10-16T08:35:24.631 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-10-16T08:35:24.672 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-10-16T08:35:24.739 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-wcwidth. 2023-10-16T08:35:24.763 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-10-16T08:35:24.798 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-16T08:35:24.974 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libradosstriper1. 2023-10-16T08:35:25.001 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../024-libradosstriper1_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:25.041 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libradosstriper1 (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:25.128 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cmd2. 2023-10-16T08:35:25.154 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-10-16T08:35:25.194 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-10-16T08:35:25.461 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-common. 2023-10-16T08:35:25.488 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../025-ceph-common_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:25.505 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libfmt8:amd64. 2023-10-16T08:35:25.527 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-common (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:25.532 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-10-16T08:35:25.614 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-16T08:35:25.958 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-10-16T08:35:25.985 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-10-16T08:35:26.025 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-16T08:35:26.320 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-10-16T08:35:26.347 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-10-16T08:35:26.379 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-16T08:35:26.698 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-10-16T08:35:26.725 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-10-16T08:35:26.765 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-16T08:35:26.775 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-base. 2023-10-16T08:35:26.801 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../026-ceph-base_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:26.851 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-base (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:27.227 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-10-16T08:35:27.254 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-16T08:35:27.302 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:35:27.579 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-10-16T08:35:27.606 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-10-16T08:35:27.638 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-10-16T08:35:27.645 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-10-16T08:35:27.665 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-16T08:35:27.697 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:35:27.906 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cheroot. 2023-10-16T08:35:27.922 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-10-16T08:35:27.965 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-16T08:35:28.075 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-10-16T08:35:28.102 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-10-16T08:35:28.141 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:35:28.260 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-10-16T08:35:28.286 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-10-16T08:35:28.326 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-10-16T08:35:28.495 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-10-16T08:35:28.522 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-10-16T08:35:28.561 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-16T08:35:28.629 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.text. 2023-10-16T08:35:28.656 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-10-16T08:35:28.695 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-10-16T08:35:28.839 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package librados2. 2023-10-16T08:35:28.855 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../011-librados2_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:28.889 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librados2 (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:28.957 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-10-16T08:35:28.972 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-10-16T08:35:29.006 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-10-16T08:35:29.285 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-tempora. 2023-10-16T08:35:29.311 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-10-16T08:35:29.351 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-10-16T08:35:29.492 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package librbd1. 2023-10-16T08:35:29.519 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../012-librbd1_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:29.558 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librbd1 (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:29.654 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-portend. 2023-10-16T08:35:29.681 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-10-16T08:35:29.720 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-10-16T08:35:29.995 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libcephfs2. 2023-10-16T08:35:30.006 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-10-16T08:35:30.011 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../013-libcephfs2_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:30.022 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-10-16T08:35:30.053 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libcephfs2 (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:30.056 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-10-16T08:35:30.314 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-rados. 2023-10-16T08:35:30.317 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-16T08:35:30.330 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../014-python3-rados_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:30.333 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-10-16T08:35:30.363 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-rados (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:30.376 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-10-16T08:35:30.625 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-10-16T08:35:30.641 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.0.0-6675-gc7471068-1jammy_all.deb ... 2023-10-16T08:35:30.674 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-ceph-argparse (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:30.721 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-natsort. 2023-10-16T08:35:30.748 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-10-16T08:35:30.788 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-10-16T08:35:30.935 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cephfs. 2023-10-16T08:35:30.951 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../016-python3-cephfs_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:31.002 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cephfs (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:31.083 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-logutils. 2023-10-16T08:35:31.109 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-10-16T08:35:31.141 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-10-16T08:35:31.313 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-ceph-common. 2023-10-16T08:35:31.341 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../017-python3-ceph-common_18.0.0-6675-gc7471068-1jammy_all.deb ... 2023-10-16T08:35:31.380 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-ceph-common (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:31.427 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-16T08:35:31.454 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-10-16T08:35:31.493 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-10-16T08:35:31.700 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-prettytable. 2023-10-16T08:35:31.727 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-10-16T08:35:31.746 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-mako. 2023-10-16T08:35:31.762 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-10-16T08:35:31.766 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-10-16T08:35:31.796 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-16T08:35:31.994 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-rbd. 2023-10-16T08:35:32.010 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../019-python3-rbd_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:32.061 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-rbd (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:32.135 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-16T08:35:32.161 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-16T08:35:32.218 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-16T08:35:32.405 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-10-16T08:35:32.433 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-10-16T08:35:32.472 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-16T08:35:32.520 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-16T08:35:32.547 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-10-16T08:35:32.587 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-10-16T08:35:32.800 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package librgw2. 2023-10-16T08:35:32.827 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../021-librgw2_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:32.866 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking librgw2 (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:32.940 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-webob. 2023-10-16T08:35:32.967 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-10-16T08:35:33.006 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-10-16T08:35:33.326 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-waitress. 2023-10-16T08:35:33.353 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-10-16T08:35:33.387 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-16T08:35:33.402 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-rgw. 2023-10-16T08:35:33.429 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../022-python3-rgw_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:33.469 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-rgw (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:33.637 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-tempita. 2023-10-16T08:35:33.650 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-10-16T08:35:33.678 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-10-16T08:35:33.789 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package liboath0:amd64. 2023-10-16T08:35:33.816 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-10-16T08:35:33.855 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-10-16T08:35:33.973 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-paste. 2023-10-16T08:35:34.000 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-10-16T08:35:34.039 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-10-16T08:35:34.183 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libradosstriper1. 2023-10-16T08:35:34.210 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../024-libradosstriper1_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:34.249 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libradosstriper1 (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:34.326 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-16T08:35:34.341 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-10-16T08:35:34.376 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-10-16T08:35:34.662 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-common. 2023-10-16T08:35:34.687 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-16T08:35:34.689 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../025-ceph-common_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:34.714 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-10-16T08:35:34.728 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-common (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:34.753 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-10-16T08:35:35.056 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-webtest. 2023-10-16T08:35:35.083 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-10-16T08:35:35.123 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-10-16T08:35:35.442 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pecan. 2023-10-16T08:35:35.469 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-10-16T08:35:35.509 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-10-16T08:35:35.872 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-certifi. 2023-10-16T08:35:35.898 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-10-16T08:35:35.938 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-10-16T08:35:36.067 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-base. 2023-10-16T08:35:36.094 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../026-ceph-base_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:36.143 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-base (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:36.224 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-idna. 2023-10-16T08:35:36.250 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-10-16T08:35:36.282 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-idna (3.3-1) ... 2023-10-16T08:35:36.560 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-urllib3. 2023-10-16T08:35:36.587 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-10-16T08:35:36.626 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-10-16T08:35:36.863 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-10-16T08:35:36.890 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-10-16T08:35:36.921 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-requests. 2023-10-16T08:35:36.930 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-10-16T08:35:36.948 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-10-16T08:35:36.987 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-16T08:35:37.182 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cheroot. 2023-10-16T08:35:37.198 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-10-16T08:35:37.241 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-16T08:35:37.299 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-16T08:35:37.326 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-10-16T08:35:37.365 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-16T08:35:37.552 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-10-16T08:35:37.579 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-10-16T08:35:37.619 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-10-16T08:35:37.635 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-16T08:35:37.650 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.0.0-6675-gc7471068-1jammy_all.deb ... 2023-10-16T08:35:37.693 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-modules-core (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:37.905 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-jaraco.text. 2023-10-16T08:35:37.932 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-10-16T08:35:37.971 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-10-16T08:35:38.274 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-10-16T08:35:38.302 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-10-16T08:35:38.341 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-10-16T08:35:38.640 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-16T08:35:38.666 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:38.710 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-tempora. 2023-10-16T08:35:38.737 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-10-16T08:35:38.794 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-10-16T08:35:38.814 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:39.097 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-portend. 2023-10-16T08:35:39.124 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-10-16T08:35:39.126 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-openssl. 2023-10-16T08:35:39.152 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-10-16T08:35:39.163 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-10-16T08:35:39.192 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-10-16T08:35:39.466 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-10-16T08:35:39.493 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-10-16T08:35:39.528 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr. 2023-10-16T08:35:39.555 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../058-ceph-mgr_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:39.595 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:39.678 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-10-16T08:35:39.948 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mon. 2023-10-16T08:35:39.969 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cherrypy3. 2023-10-16T08:35:39.970 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../059-ceph-mon_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:39.985 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-10-16T08:35:40.006 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mon (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:40.019 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-10-16T08:35:40.885 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-osd. 2023-10-16T08:35:40.911 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../060-ceph-osd_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:40.951 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-osd (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:41.031 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-natsort. 2023-10-16T08:35:41.058 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-10-16T08:35:41.097 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-10-16T08:35:41.417 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-logutils. 2023-10-16T08:35:41.445 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-10-16T08:35:41.484 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-10-16T08:35:41.770 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-markupsafe. 2023-10-16T08:35:41.797 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-10-16T08:35:41.837 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-10-16T08:35:42.140 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-mako. 2023-10-16T08:35:42.167 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-10-16T08:35:42.206 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-16T08:35:42.560 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-simplegeneric. 2023-10-16T08:35:42.586 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-10-16T08:35:42.626 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-10-16T08:35:42.896 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-singledispatch. 2023-10-16T08:35:42.922 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-10-16T08:35:42.962 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-10-16T08:35:43.154 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph. 2023-10-16T08:35:43.180 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../061-ceph_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:43.221 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:43.248 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-webob. 2023-10-16T08:35:43.275 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-10-16T08:35:43.315 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-10-16T08:35:43.540 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-fuse. 2023-10-16T08:35:43.566 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../062-ceph-fuse_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:43.606 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-fuse (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:43.668 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-waitress. 2023-10-16T08:35:43.695 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-10-16T08:35:43.738 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-16T08:35:43.943 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mds. 2023-10-16T08:35:43.969 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../063-ceph-mds_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:44.009 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mds (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:44.021 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-tempita. 2023-10-16T08:35:44.048 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-10-16T08:35:44.088 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-10-16T08:35:44.379 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephadm. 2023-10-16T08:35:44.399 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-paste. 2023-10-16T08:35:44.405 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../064-cephadm_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:44.426 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-10-16T08:35:44.445 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephadm (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:44.465 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-10-16T08:35:44.731 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-asyncssh. 2023-10-16T08:35:44.758 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-10-16T08:35:44.760 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-10-16T08:35:44.776 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-10-16T08:35:44.798 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-10-16T08:35:44.818 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-10-16T08:35:45.142 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python-babel-localedata. 2023-10-16T08:35:45.169 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-10-16T08:35:45.234 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-16T08:35:45.263 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pastedeploy. 2023-10-16T08:35:45.290 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-10-16T08:35:45.330 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-10-16T08:35:45.616 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-webtest. 2023-10-16T08:35:45.643 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-10-16T08:35:45.683 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-10-16T08:35:45.994 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pecan. 2023-10-16T08:35:46.021 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-10-16T08:35:46.061 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-10-16T08:35:46.172 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-babel. 2023-10-16T08:35:46.187 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-10-16T08:35:46.221 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-10-16T08:35:46.399 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-certifi. 2023-10-16T08:35:46.426 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-10-16T08:35:46.457 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-10-16T08:35:46.491 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-jinja2. 2023-10-16T08:35:46.506 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-10-16T08:35:46.541 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-10-16T08:35:46.777 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-idna. 2023-10-16T08:35:46.804 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-10-16T08:35:46.843 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-idna (3.3-1) ... 2023-10-16T08:35:46.844 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-16T08:35:46.871 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.0.0-6675-gc7471068-1jammy_all.deb ... 2023-10-16T08:35:46.911 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-cephadm (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:47.155 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-urllib3. 2023-10-16T08:35:47.182 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1_all.deb ... 2023-10-16T08:35:47.206 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-16T08:35:47.221 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1) ... 2023-10-16T08:35:47.232 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-10-16T08:35:47.272 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-10-16T08:35:47.525 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-requests. 2023-10-16T08:35:47.552 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-10-16T08:35:47.567 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-16T08:35:47.591 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-16T08:35:47.593 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../071-python3-repoze.lru_0.7-2_all.deb ... 2023-10-16T08:35:47.633 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-16T08:35:47.902 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-werkzeug. 2023-10-16T08:35:47.929 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-10-16T08:35:47.936 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-routes. 2023-10-16T08:35:47.963 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../072-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-10-16T08:35:47.969 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-16T08:35:48.003 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-10-16T08:35:48.280 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-10-16T08:35:48.297 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.0.0-6675-gc7471068-1jammy_all.deb ... 2023-10-16T08:35:48.339 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-modules-core (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:48.339 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-16T08:35:48.366 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../073-ceph-mgr-dashboard_18.0.0-6675-gc7471068-1jammy_all.deb ... 2023-10-16T08:35:48.405 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-dashboard (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:48.710 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-10-16T08:35:48.737 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:48.777 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libsqlite3-mod-ceph (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:48.817 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-10-16T08:35:48.844 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-10-16T08:35:48.884 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-16T08:35:49.088 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-openssl. 2023-10-16T08:35:49.115 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-10-16T08:35:49.154 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-10-16T08:35:49.396 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-joblib. 2023-10-16T08:35:49.422 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../075-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-10-16T08:35:49.462 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-10-16T08:35:49.491 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr. 2023-10-16T08:35:49.518 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../058-ceph-mgr_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:49.566 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:49.807 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-10-16T08:35:49.834 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../076-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-10-16T08:35:49.873 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-10-16T08:35:49.911 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mon. 2023-10-16T08:35:49.938 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../059-ceph-mon_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:49.969 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mon (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:50.177 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-sklearn. 2023-10-16T08:35:50.203 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../077-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-10-16T08:35:50.243 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-16T08:35:50.865 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-osd. 2023-10-16T08:35:50.891 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../060-ceph-osd_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:50.931 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-osd (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:50.968 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-16T08:35:50.995 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../078-ceph-mgr-diskprediction-local_18.0.0-6675-gc7471068-1jammy_all.deb ... 2023-10-16T08:35:51.034 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:51.929 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-cachetools. 2023-10-16T08:35:51.956 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../079-python3-cachetools_5.0.0-1_all.deb ... 2023-10-16T08:35:51.995 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-10-16T08:35:52.256 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-16T08:35:52.280 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../080-python3-pyasn1_0.4.8-1_all.deb ... 2023-10-16T08:35:52.314 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-10-16T08:35:52.651 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-16T08:35:52.678 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../081-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-10-16T08:35:52.717 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-10-16T08:35:52.934 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph. 2023-10-16T08:35:52.950 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../061-ceph_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:52.992 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:53.012 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-rsa. 2023-10-16T08:35:53.039 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../082-python3-rsa_4.8-1_all.deb ... 2023-10-16T08:35:53.078 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-rsa (4.8-1) ... 2023-10-16T08:35:53.279 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-fuse. 2023-10-16T08:35:53.305 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../062-ceph-fuse_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:53.345 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-fuse (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:53.683 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-google-auth. 2023-10-16T08:35:53.710 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../083-python3-google-auth_1.5.1-3_all.deb ... 2023-10-16T08:35:53.715 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mds. 2023-10-16T08:35:53.742 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../063-ceph-mds_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:53.749 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-10-16T08:35:53.782 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mds (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:54.069 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-16T08:35:54.096 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../084-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-10-16T08:35:54.136 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-16T08:35:54.168 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package cephadm. 2023-10-16T08:35:54.195 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../064-cephadm_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:54.235 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking cephadm (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:54.414 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-websocket. 2023-10-16T08:35:54.429 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../085-python3-websocket_1.2.3-1_all.deb ... 2023-10-16T08:35:54.464 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-10-16T08:35:54.555 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-asyncssh. 2023-10-16T08:35:54.582 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-10-16T08:35:54.621 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-10-16T08:35:54.742 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-16T08:35:54.768 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../086-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-10-16T08:35:54.808 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-16T08:35:54.933 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python-babel-localedata. 2023-10-16T08:35:54.960 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-10-16T08:35:54.999 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-16T08:35:55.473 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-16T08:35:55.499 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../087-ceph-mgr-k8sevents_18.0.0-6675-gc7471068-1jammy_all.deb ... 2023-10-16T08:35:55.538 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:55.816 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-16T08:35:55.843 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../088-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-10-16T08:35:55.874 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-16T08:35:56.005 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-babel. 2023-10-16T08:35:56.021 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-10-16T08:35:56.054 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-10-16T08:35:56.238 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-16T08:35:56.263 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../089-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-10-16T08:35:56.302 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-16T08:35:56.391 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-jinja2. 2023-10-16T08:35:56.418 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-10-16T08:35:56.457 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-10-16T08:35:56.562 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package jq. 2023-10-16T08:35:56.578 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../090-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-10-16T08:35:56.621 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-10-16T08:35:56.828 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-10-16T08:35:56.855 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.0.0-6675-gc7471068-1jammy_all.deb ... 2023-10-16T08:35:56.873 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package socat. 2023-10-16T08:35:56.889 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../091-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-10-16T08:35:56.894 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-cephadm (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:56.932 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-10-16T08:35:57.215 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-bcrypt. 2023-10-16T08:35:57.226 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package xmlstarlet. 2023-10-16T08:35:57.242 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-10-16T08:35:57.253 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../092-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-10-16T08:35:57.281 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-10-16T08:35:57.284 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-10-16T08:35:57.542 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-repoze.lru. 2023-10-16T08:35:57.569 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../071-python3-repoze.lru_0.7-2_all.deb ... 2023-10-16T08:35:57.609 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-10-16T08:35:57.621 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-test. 2023-10-16T08:35:57.637 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../093-ceph-test_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:35:57.671 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-test (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:57.912 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-routes. 2023-10-16T08:35:57.939 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../072-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-10-16T08:35:57.978 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-10-16T08:35:58.274 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-10-16T08:35:58.290 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../073-ceph-mgr-dashboard_18.0.0-6675-gc7471068-1jammy_all.deb ... 2023-10-16T08:35:58.324 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-dashboard (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:35:58.752 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-10-16T08:35:58.768 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../074-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-10-16T08:35:58.810 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-16T08:35:59.298 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-joblib. 2023-10-16T08:35:59.325 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../075-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-10-16T08:35:59.397 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-10-16T08:35:59.734 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-10-16T08:35:59.750 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../076-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-10-16T08:35:59.784 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-10-16T08:36:00.017 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ceph-volume. 2023-10-16T08:36:00.044 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../094-ceph-volume_18.0.0-6675-gc7471068-1jammy_all.deb ... 2023-10-16T08:36:00.079 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-sklearn. 2023-10-16T08:36:00.083 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ceph-volume (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:00.106 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../077-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-10-16T08:36:00.145 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-16T08:36:00.429 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-16T08:36:00.456 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../095-cephfs-mirror_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:36:00.495 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephfs-mirror (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:00.807 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-colorama. 2023-10-16T08:36:00.834 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../096-python3-colorama_0.4.4-1_all.deb ... 2023-10-16T08:36:00.846 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-10-16T08:36:00.873 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-10-16T08:36:00.874 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../078-ceph-mgr-diskprediction-local_18.0.0-6675-gc7471068-1jammy_all.deb ... 2023-10-16T08:36:00.904 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-diskprediction-local (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:01.143 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephfs-shell. 2023-10-16T08:36:01.170 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../097-cephfs-shell_18.0.0-6675-gc7471068-1jammy_all.deb ... 2023-10-16T08:36:01.209 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephfs-shell (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:01.445 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package cephfs-top. 2023-10-16T08:36:01.461 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../098-cephfs-top_18.0.0-6675-gc7471068-1jammy_all.deb ... 2023-10-16T08:36:01.495 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking cephfs-top (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:01.731 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-16T08:36:01.746 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../099-libcephfs-dev_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:36:01.789 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libcephfs-dev (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:01.841 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-cachetools. 2023-10-16T08:36:01.868 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../079-python3-cachetools_5.0.0-1_all.deb ... 2023-10-16T08:36:01.907 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-10-16T08:36:02.159 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package nvme-cli. 2023-10-16T08:36:02.186 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../100-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-10-16T08:36:02.210 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pyasn1. 2023-10-16T08:36:02.225 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-10-16T08:36:02.227 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../080-python3-pyasn1_0.4.8-1_all.deb ... 2023-10-16T08:36:02.268 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-10-16T08:36:02.546 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-10-16T08:36:02.574 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../101-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-10-16T08:36:02.605 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-10-16T08:36:02.613 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-10-16T08:36:02.632 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../081-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-10-16T08:36:02.671 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-10-16T08:36:02.958 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-iniconfig. 2023-10-16T08:36:02.983 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-rsa. 2023-10-16T08:36:02.985 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../102-python3-iniconfig_1.1.1-2_all.deb ... 2023-10-16T08:36:03.010 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../082-python3-rsa_4.8-1_all.deb ... 2023-10-16T08:36:03.015 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-10-16T08:36:03.049 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-rsa (4.8-1) ... 2023-10-16T08:36:03.352 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pastescript. 2023-10-16T08:36:03.379 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-4_all.deb ... 2023-10-16T08:36:03.418 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-16T08:36:03.438 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-google-auth. 2023-10-16T08:36:03.465 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../083-python3-google-auth_1.5.1-3_all.deb ... 2023-10-16T08:36:03.504 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-10-16T08:36:03.738 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pluggy. 2023-10-16T08:36:03.765 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../104-python3-pluggy_0.13.0-7.1_all.deb ... 2023-10-16T08:36:03.804 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-10-16T08:36:03.855 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-10-16T08:36:03.883 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../084-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-10-16T08:36:03.926 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-16T08:36:04.116 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-psutil. 2023-10-16T08:36:04.143 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../105-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-10-16T08:36:04.182 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-10-16T08:36:04.238 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-websocket. 2023-10-16T08:36:04.265 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../085-python3-websocket_1.2.3-1_all.deb ... 2023-10-16T08:36:04.304 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-10-16T08:36:04.519 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-py. 2023-10-16T08:36:04.546 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../106-python3-py_1.10.0-1_all.deb ... 2023-10-16T08:36:04.585 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-py (1.10.0-1) ... 2023-10-16T08:36:04.616 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-kubernetes. 2023-10-16T08:36:04.643 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../086-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-10-16T08:36:04.682 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-16T08:36:04.905 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pygments. 2023-10-16T08:36:04.932 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../107-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-10-16T08:36:04.971 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-10-16T08:36:05.321 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-10-16T08:36:05.338 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../087-ceph-mgr-k8sevents_18.0.0-6675-gc7471068-1jammy_all.deb ... 2023-10-16T08:36:05.392 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-16T08:36:05.395 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-mgr-k8sevents (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:05.419 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../108-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-10-16T08:36:05.458 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-10-16T08:36:05.707 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libonig5:amd64. 2023-10-16T08:36:05.735 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../088-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-10-16T08:36:05.744 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-toml. 2023-10-16T08:36:05.771 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../109-python3-toml_0.10.2-1_all.deb ... 2023-10-16T08:36:05.773 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-16T08:36:05.810 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-10-16T08:36:06.097 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-pytest. 2023-10-16T08:36:06.124 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../110-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-10-16T08:36:06.163 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-10-16T08:36:06.335 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libjq1:amd64. 2023-10-16T08:36:06.363 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../089-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-10-16T08:36:06.401 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-16T08:36:06.508 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package python3-simplejson. 2023-10-16T08:36:06.535 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-10-16T08:36:06.574 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-10-16T08:36:06.654 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package jq. 2023-10-16T08:36:06.682 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../090-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-10-16T08:36:06.720 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-10-16T08:36:06.877 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-16T08:36:06.893 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-10-16T08:36:06.927 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-10-16T08:36:06.931 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package socat. 2023-10-16T08:36:06.948 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../091-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-10-16T08:36:06.990 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-10-16T08:36:07.285 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package xmlstarlet. 2023-10-16T08:36:07.301 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../092-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-10-16T08:36:07.343 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-10-16T08:36:07.639 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package radosgw. 2023-10-16T08:36:07.666 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../113-radosgw_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:36:07.705 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking radosgw (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:07.722 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-test. 2023-10-16T08:36:07.750 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../093-ceph-test_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:36:07.788 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-test (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:08.458 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package rbd-fuse. 2023-10-16T08:36:08.486 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../114-rbd-fuse_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:36:08.524 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking rbd-fuse (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:08.829 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-10-16T08:36:09.079 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-16T08:36:09.205 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-10-16T08:36:09.469 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-10-16T08:36:09.842 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-10-16T08:36:10.276 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ceph-volume. 2023-10-16T08:36:10.304 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../094-ceph-volume_18.0.0-6675-gc7471068-1jammy_all.deb ... 2023-10-16T08:36:10.304 INFO:teuthology.orchestra.run.smithi086.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-10-16T08:36:10.342 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ceph-volume (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:10.373 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephadm (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:10.680 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package cephfs-mirror. 2023-10-16T08:36:10.708 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../095-cephfs-mirror_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:36:10.746 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking cephfs-mirror (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:10.915 INFO:teuthology.orchestra.run.smithi086.stdout:Adding system user cephadm....done 2023-10-16T08:36:11.071 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-16T08:36:11.091 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-colorama. 2023-10-16T08:36:11.119 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../096-python3-colorama_0.4.4-1_all.deb ... 2023-10-16T08:36:11.158 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-10-16T08:36:11.355 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-10-16T08:36:11.436 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package cephfs-shell. 2023-10-16T08:36:11.461 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../097-cephfs-shell_18.0.0-6675-gc7471068-1jammy_all.deb ... 2023-10-16T08:36:11.502 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking cephfs-shell (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:11.606 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-10-16T08:36:11.730 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package cephfs-top. 2023-10-16T08:36:11.732 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-10-16T08:36:11.746 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../098-cephfs-top_18.0.0-6675-gc7471068-1jammy_all.deb ... 2023-10-16T08:36:11.780 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking cephfs-top (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:11.983 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-16T08:36:11.999 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libcephfs-dev. 2023-10-16T08:36:12.022 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../099-libcephfs-dev_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:36:12.066 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libcephfs-dev (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:12.235 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-10-16T08:36:12.336 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-py (1.10.0-1) ... 2023-10-16T08:36:12.444 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package nvme-cli. 2023-10-16T08:36:12.472 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../100-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-10-16T08:36:12.510 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-10-16T08:36:12.620 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-10-16T08:36:12.849 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-10-16T08:36:12.865 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../101-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-10-16T08:36:12.915 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-10-16T08:36:12.981 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-10-16T08:36:13.248 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-10-16T08:36:13.327 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-iniconfig. 2023-10-16T08:36:13.354 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../102-python3-iniconfig_1.1.1-2_all.deb ... 2023-10-16T08:36:13.385 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-10-16T08:36:13.499 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-10-16T08:36:13.654 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pastescript. 2023-10-16T08:36:13.682 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../103-python3-pastescript_2.0.2-4_all.deb ... 2023-10-16T08:36:13.721 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-10-16T08:36:13.767 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-10-16T08:36:13.995 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-10-16T08:36:14.024 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pluggy. 2023-10-16T08:36:14.054 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../104-python3-pluggy_0.13.0-7.1_all.deb ... 2023-10-16T08:36:14.082 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-10-16T08:36:14.255 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-10-16T08:36:14.369 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-psutil. 2023-10-16T08:36:14.396 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../105-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-10-16T08:36:14.435 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-10-16T08:36:14.506 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-ceph-argparse (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:14.765 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-16T08:36:14.780 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-py. 2023-10-16T08:36:14.807 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../106-python3-py_1.10.0-1_all.deb ... 2023-10-16T08:36:14.846 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-py (1.10.0-1) ... 2023-10-16T08:36:14.891 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-10-16T08:36:15.150 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-10-16T08:36:15.233 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pygments. 2023-10-16T08:36:15.261 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../107-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-10-16T08:36:15.300 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-10-16T08:36:15.393 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-16T08:36:15.494 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-10-16T08:36:15.746 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pyinotify. 2023-10-16T08:36:15.774 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../108-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-10-16T08:36:15.803 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-10-16T08:36:15.836 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-16T08:36:15.962 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-10-16T08:36:16.115 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-toml. 2023-10-16T08:36:16.143 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../109-python3-toml_0.10.2-1_all.deb ... 2023-10-16T08:36:16.181 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-10-16T08:36:16.230 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-10-16T08:36:16.498 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-10-16T08:36:16.709 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-pytest. 2023-10-16T08:36:16.738 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../110-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-10-16T08:36:16.776 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-10-16T08:36:16.807 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-10-16T08:36:17.121 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package python3-simplejson. 2023-10-16T08:36:17.149 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-10-16T08:36:17.187 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-10-16T08:36:17.457 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-10-16T08:36:17.468 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-10-16T08:36:17.484 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-10-16T08:36:17.523 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-10-16T08:36:17.710 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-10-16T08:36:17.820 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-10-16T08:36:17.954 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-16T08:36:18.255 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-10-16T08:36:18.277 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package radosgw. 2023-10-16T08:36:18.305 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../113-radosgw_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:36:18.343 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking radosgw (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:18.664 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-10-16T08:36:18.915 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-10-16T08:36:19.171 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package rbd-fuse. 2023-10-16T08:36:19.189 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../114-rbd-fuse_18.0.0-6675-gc7471068-1jammy_amd64.deb ... 2023-10-16T08:36:19.229 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking rbd-fuse (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:19.309 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-16T08:36:19.518 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-10-16T08:36:19.577 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-10-16T08:36:19.776 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-10-16T08:36:19.852 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-16T08:36:19.902 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-10-16T08:36:20.162 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-10-16T08:36:20.196 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-10-16T08:36:20.447 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-10-16T08:36:20.536 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-10-16T08:36:20.573 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-ceph-common (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:20.866 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-idna (3.3-1) ... 2023-10-16T08:36:21.003 INFO:teuthology.orchestra.run.smithi040.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-10-16T08:36:21.112 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up cephadm (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:21.216 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-16T08:36:21.631 INFO:teuthology.orchestra.run.smithi040.stdout:Adding system user cephadm....done 2023-10-16T08:36:21.810 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-10-16T08:36:22.128 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-10-16T08:36:22.322 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:36:22.404 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-10-16T08:36:22.456 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-toml (0.10.2-1) ... 2023-10-16T08:36:22.530 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-10-16T08:36:22.716 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-16T08:36:22.773 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-10-16T08:36:22.833 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-10-16T08:36:22.934 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-10-16T08:36:23.025 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-10-16T08:36:23.151 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-py (1.10.0-1) ... 2023-10-16T08:36:23.202 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-10-16T08:36:23.452 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-10-16T08:36:23.453 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-10-16T08:36:23.713 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:36:23.837 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-10-16T08:36:23.838 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-16T08:36:23.965 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-10-16T08:36:24.105 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-10-16T08:36:24.383 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-10-16T08:36:24.390 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-10-16T08:36:24.650 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-10-16T08:36:24.658 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-10-16T08:36:24.893 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-10-16T08:36:24.909 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-10-16T08:36:25.152 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-16T08:36:25.185 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-10-16T08:36:25.395 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-10-16T08:36:25.453 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-ceph-argparse (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:25.646 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-16T08:36:25.720 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-10-16T08:36:25.772 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-10-16T08:36:25.846 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-10-16T08:36:26.056 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-16T08:36:26.114 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-10-16T08:36:26.183 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-16T08:36:26.382 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-10-16T08:36:26.467 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-10-16T08:36:26.483 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-10-16T08:36:26.751 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-10-16T08:36:26.876 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-10-16T08:36:26.986 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-10-16T08:36:27.002 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-10-16T08:36:27.278 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-10-16T08:36:27.382 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-10-16T08:36:27.554 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-10-16T08:36:27.647 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-16T08:36:27.773 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-10-16T08:36:27.863 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-10-16T08:36:28.166 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-10-16T08:36:28.450 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-portend (3.0.0-1) ... 2023-10-16T08:36:28.599 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-10-16T08:36:28.684 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:36:28.794 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-10-16T08:36:28.875 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-10-16T08:36:29.026 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-10-16T08:36:29.103 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-10-16T08:36:29.152 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-10-16T08:36:29.396 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-16T08:36:29.453 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-10-16T08:36:29.672 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-10-16T08:36:29.798 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rsa (4.8-1) ... 2023-10-16T08:36:29.854 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-10-16T08:36:30.074 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-10-16T08:36:30.139 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-10-16T08:36:30.342 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-16T08:36:30.565 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-10-16T08:36:30.585 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-10-16T08:36:30.867 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-10-16T08:36:30.953 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-16T08:36:31.201 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-10-16T08:36:31.245 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-10-16T08:36:31.552 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-10-16T08:36:31.589 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-16T08:36:31.715 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librados2 (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:31.812 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-10-16T08:36:31.858 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librgw2 (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:31.938 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-ceph-common (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:32.052 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libsqlite3-mod-ceph (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:32.178 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libcephfs2 (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:32.304 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libradosstriper1 (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:32.422 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-idna (3.3-1) ... 2023-10-16T08:36:32.430 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-10-16T08:36:32.707 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up librbd1 (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:32.773 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-10-16T08:36:32.833 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-modules-core (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:32.942 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-fuse (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:33.137 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-16T08:36:33.137 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-16T08:36:33.660 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libcephfs-dev (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:33.761 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rados (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:33.887 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephfs-top (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:33.963 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:36:34.013 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-16T08:36:34.097 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-toml (0.10.2-1) ... 2023-10-16T08:36:34.365 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-10-16T08:36:34.491 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-10-16T08:36:34.617 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-urllib3 (1.26.5-1~exp1) ... 2023-10-16T08:36:34.918 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-10-16T08:36:35.175 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rbd (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:35.178 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-10-16T08:36:35.276 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up rbd-fuse (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:35.394 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-rgw (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:35.437 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:36:35.520 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up python3-cephfs (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:35.588 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-10-16T08:36:35.654 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephfs-shell (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:35.739 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-10-16T08:36:35.780 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-common (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:36.041 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-10-16T08:36:36.317 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-10-16T08:36:36.331 INFO:teuthology.orchestra.run.smithi086.stdout:Adding group ceph....done 2023-10-16T08:36:36.576 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-10-16T08:36:36.666 INFO:teuthology.orchestra.run.smithi086.stdout:Adding system user ceph....done 2023-10-16T08:36:36.836 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-10-16T08:36:36.851 INFO:teuthology.orchestra.run.smithi086.stdout:Setting system user ceph properties....done 2023-10-16T08:36:36.868 INFO:teuthology.orchestra.run.smithi086.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-16T08:36:37.015 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-16T08:36:37.095 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-10-16T08:36:37.357 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-16T08:36:37.396 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-10-16T08:36:37.522 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-10-16T08:36:37.799 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-10-16T08:36:37.916 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-10-16T08:36:37.982 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up cephfs-mirror (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:38.200 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-16T08:36:38.201 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-16T08:36:38.226 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-10-16T08:36:38.535 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-10-16T08:36:38.691 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-test (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:38.800 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up radosgw (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:38.803 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-10-16T08:36:39.202 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-10-16T08:36:39.276 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-16T08:36:39.276 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-16T08:36:39.464 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-10-16T08:36:39.590 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-10-16T08:36:39.855 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-base (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:40.025 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-10-16T08:36:40.193 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-16T08:36:40.293 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-portend (3.0.0-1) ... 2023-10-16T08:36:40.553 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-10-16T08:36:40.678 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-10-16T08:36:40.724 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mds (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:40.947 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-16T08:36:40.947 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-16T08:36:40.996 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-10-16T08:36:41.322 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-10-16T08:36:41.483 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:41.581 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-10-16T08:36:41.708 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-16T08:36:41.708 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-16T08:36:41.716 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-rsa (4.8-1) ... 2023-10-16T08:36:41.993 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-10-16T08:36:42.234 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-osd (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:42.268 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-10-16T08:36:42.517 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-16T08:36:42.518 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-16T08:36:42.537 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-10-16T08:36:42.963 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-10-16T08:36:43.069 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-k8sevents (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:43.187 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:43.275 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-10-16T08:36:43.338 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mon (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:43.556 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-16T08:36:43.556 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-16T08:36:43.625 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-10-16T08:36:43.751 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librados2 (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:43.877 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librgw2 (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:44.089 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-cephadm (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:44.096 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libsqlite3-mod-ceph (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:44.214 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libcephfs2 (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:44.240 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:44.315 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libradosstriper1 (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:44.366 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-mgr-dashboard (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:44.425 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-10-16T08:36:44.517 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ceph-volume (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:44.702 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up librbd1 (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:44.828 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-modules-core (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:44.978 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-fuse (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:45.048 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-16T08:36:45.218 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-16T08:36:45.218 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-10-16T08:36:45.738 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libcephfs-dev (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:45.881 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-rados (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:46.007 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up cephfs-top (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:46.117 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-16T08:36:46.125 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-10-16T08:36:46.569 INFO:teuthology.orchestra.run.smithi086.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-16T08:36:46.572 DEBUG:teuthology.orchestra.run.smithi086:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-10-16T08:36:46.624 INFO:teuthology.orchestra.run.smithi086.stdout:Reading package lists... 2023-10-16T08:36:46.808 INFO:teuthology.orchestra.run.smithi086.stdout:Building dependency tree... 2023-10-16T08:36:46.809 INFO:teuthology.orchestra.run.smithi086.stdout:Reading state information... 2023-10-16T08:36:46.972 INFO:teuthology.orchestra.run.smithi086.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-10-16T08:36:46.973 INFO:teuthology.orchestra.run.smithi086.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-10-16T08:36:46.973 INFO:teuthology.orchestra.run.smithi086.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-10-16T08:36:46.973 INFO:teuthology.orchestra.run.smithi086.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-10-16T08:36:46.973 INFO:teuthology.orchestra.run.smithi086.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-10-16T08:36:46.974 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages were automatically installed and are no longer required: 2023-10-16T08:36:46.974 INFO:teuthology.orchestra.run.smithi086.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-16T08:36:46.974 INFO:teuthology.orchestra.run.smithi086.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-16T08:36:46.974 INFO:teuthology.orchestra.run.smithi086.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-16T08:36:46.974 INFO:teuthology.orchestra.run.smithi086.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-16T08:36:46.976 INFO:teuthology.orchestra.run.smithi086.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-16T08:36:46.976 INFO:teuthology.orchestra.run.smithi086.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-16T08:36:46.976 INFO:teuthology.orchestra.run.smithi086.stdout:The following additional packages will be installed: 2023-10-16T08:36:46.976 INFO:teuthology.orchestra.run.smithi086.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-10-16T08:36:46.976 INFO:teuthology.orchestra.run.smithi086.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-10-16T08:36:46.977 INFO:teuthology.orchestra.run.smithi086.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-10-16T08:36:46.977 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-10-16T08:36:46.977 INFO:teuthology.orchestra.run.smithi086.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-16T08:36:46.977 INFO:teuthology.orchestra.run.smithi086.stdout:Suggested packages: 2023-10-16T08:36:46.977 INFO:teuthology.orchestra.run.smithi086.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-10-16T08:36:46.977 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-10-16T08:36:46.978 INFO:teuthology.orchestra.run.smithi086.stdout: libengine-pkcs11-openssl 2023-10-16T08:36:47.014 INFO:teuthology.orchestra.run.smithi086.stdout:The following NEW packages will be installed: 2023-10-16T08:36:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-10-16T08:36:47.015 INFO:teuthology.orchestra.run.smithi086.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-10-16T08:36:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-10-16T08:36:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-10-16T08:36:47.016 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-10-16T08:36:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-10-16T08:36:47.017 INFO:teuthology.orchestra.run.smithi086.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-16T08:36:47.017 INFO:teuthology.orchestra.run.smithi086.stdout:The following packages will be upgraded: 2023-10-16T08:36:47.018 INFO:teuthology.orchestra.run.smithi086.stdout: libssl3 2023-10-16T08:36:47.069 INFO:teuthology.orchestra.run.smithi086.stdout:1 upgraded, 39 newly installed, 0 to remove and 255 not upgraded. 2023-10-16T08:36:47.069 INFO:teuthology.orchestra.run.smithi086.stdout:Need to get 54.6 MB of archives. 2023-10-16T08:36:47.070 INFO:teuthology.orchestra.run.smithi086.stdout:After this operation, 202 MB of additional disk space will be used. 2023-10-16T08:36:47.070 INFO:teuthology.orchestra.run.smithi086.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-10-16T08:36:47.168 INFO:teuthology.orchestra.run.smithi086.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.10 [1,901 kB] 2023-10-16T08:36:47.229 INFO:teuthology.orchestra.run.smithi086.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-10-16T08:36:47.245 INFO:teuthology.orchestra.run.smithi086.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-10-16T08:36:47.246 INFO:teuthology.orchestra.run.smithi086.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-10-16T08:36:47.247 INFO:teuthology.orchestra.run.smithi086.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-10-16T08:36:47.249 INFO:teuthology.orchestra.run.smithi086.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-10-16T08:36:47.250 INFO:teuthology.orchestra.run.smithi086.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-10-16T08:36:47.251 INFO:teuthology.orchestra.run.smithi086.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-10-16T08:36:47.253 INFO:teuthology.orchestra.run.smithi086.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-10-16T08:36:47.253 INFO:teuthology.orchestra.run.smithi086.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-10-16T08:36:47.254 INFO:teuthology.orchestra.run.smithi086.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-10-16T08:36:47.267 INFO:teuthology.orchestra.run.smithi086.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-10-16T08:36:47.371 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-rbd (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:47.522 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up rbd-fuse (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:47.560 INFO:teuthology.orchestra.run.smithi086.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-10-16T08:36:47.562 INFO:teuthology.orchestra.run.smithi086.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-10-16T08:36:47.562 INFO:teuthology.orchestra.run.smithi086.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-10-16T08:36:47.565 INFO:teuthology.orchestra.run.smithi086.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-10-16T08:36:47.565 INFO:teuthology.orchestra.run.smithi086.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-10-16T08:36:47.569 INFO:teuthology.orchestra.run.smithi086.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-10-16T08:36:47.570 INFO:teuthology.orchestra.run.smithi086.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-10-16T08:36:47.577 INFO:teuthology.orchestra.run.smithi086.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-10-16T08:36:47.577 INFO:teuthology.orchestra.run.smithi086.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.10 [2,372 kB] 2023-10-16T08:36:47.596 INFO:teuthology.orchestra.run.smithi086.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-10-16T08:36:47.596 INFO:teuthology.orchestra.run.smithi086.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-10-16T08:36:47.605 INFO:teuthology.orchestra.run.smithi086.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-10-16T08:36:47.626 INFO:teuthology.orchestra.run.smithi086.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-10-16T08:36:47.629 INFO:teuthology.orchestra.run.smithi086.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-10-16T08:36:47.629 INFO:teuthology.orchestra.run.smithi086.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-10-16T08:36:47.630 INFO:teuthology.orchestra.run.smithi086.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-10-16T08:36:47.630 INFO:teuthology.orchestra.run.smithi086.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-10-16T08:36:47.640 INFO:teuthology.orchestra.run.smithi086.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-10-16T08:36:47.642 INFO:teuthology.orchestra.run.smithi086.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-10-16T08:36:47.800 INFO:teuthology.orchestra.run.smithi086.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-10-16T08:36:47.843 INFO:teuthology.orchestra.run.smithi086.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-10-16T08:36:47.844 INFO:teuthology.orchestra.run.smithi086.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-10-16T08:36:47.844 INFO:teuthology.orchestra.run.smithi086.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-10-16T08:36:47.844 INFO:teuthology.orchestra.run.smithi086.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-10-16T08:36:47.844 INFO:teuthology.orchestra.run.smithi086.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-10-16T08:36:47.854 INFO:teuthology.orchestra.run.smithi086.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-10-16T08:36:47.855 INFO:teuthology.orchestra.run.smithi086.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-10-16T08:36:47.940 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-rgw (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:48.066 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up python3-cephfs (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:48.192 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up cephfs-shell (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:48.252 INFO:teuthology.orchestra.run.smithi086.stdout:Preconfiguring packages ... 2023-10-16T08:36:48.318 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-common (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:48.381 INFO:teuthology.orchestra.run.smithi086.stdout:Fetched 54.6 MB in 1s (65.7 MB/s) 2023-10-16T08:36:48.520 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package flex. 2023-10-16T08:36:48.584 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172907 files and directories currently installed.) 2023-10-16T08:36:48.589 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-10-16T08:36:48.636 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking flex (2.6.4-8build2) ... 2023-10-16T08:36:48.912 INFO:teuthology.orchestra.run.smithi040.stdout:Adding group ceph....done 2023-10-16T08:36:49.194 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.10_amd64.deb ... 2023-10-16T08:36:49.288 INFO:teuthology.orchestra.run.smithi040.stdout:Adding system user ceph....done 2023-10-16T08:36:49.358 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.10) over (3.0.2-0ubuntu1.7) ... 2023-10-16T08:36:49.423 INFO:teuthology.orchestra.run.smithi040.stdout:Setting system user ceph properties....done 2023-10-16T08:36:49.438 INFO:teuthology.orchestra.run.smithi040.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-10-16T08:36:49.580 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-10-16T08:36:49.767 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-16T08:36:49.926 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-10-16T08:36:50.301 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package bison. 2023-10-16T08:36:50.404 INFO:teuthology.orchestra.run.smithi086.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172980 files and directories currently installed.) 2023-10-16T08:36:50.410 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-10-16T08:36:50.461 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-10-16T08:36:50.511 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up cephfs-mirror (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:50.787 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-16T08:36:50.788 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-10-16T08:36:50.986 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-16T08:36:51.013 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-10-16T08:36:51.052 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-10-16T08:36:51.304 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-test (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:51.397 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package dump. 2023-10-16T08:36:51.422 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up radosgw (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:51.424 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-10-16T08:36:51.464 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking dump (0.4b47-3) ... 2023-10-16T08:36:51.759 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package indent. 2023-10-16T08:36:51.783 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:36:51.817 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-16T08:36:51.895 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-16T08:36:51.896 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-10-16T08:36:52.028 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-16T08:36:52.044 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-10-16T08:36:52.078 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-16T08:36:52.381 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-16T08:36:52.398 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-base (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:52.409 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-10-16T08:36:52.448 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-16T08:36:52.709 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-16T08:36:52.725 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-10-16T08:36:52.749 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-10-16T08:36:52.767 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-10-16T08:36:53.053 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libfl2:amd64. 2023-10-16T08:36:53.066 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-10-16T08:36:53.103 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-10-16T08:36:53.251 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mds (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:53.348 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-10-16T08:36:53.375 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-10-16T08:36:53.414 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-16T08:36:53.494 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-16T08:36:53.495 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-10-16T08:36:53.666 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libjson-perl. 2023-10-16T08:36:53.694 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-10-16T08:36:53.733 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-10-16T08:36:53.994 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:54.036 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libllvm14:amd64. 2023-10-16T08:36:54.064 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-10-16T08:36:54.102 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-16T08:36:54.243 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-16T08:36:54.244 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-10-16T08:36:54.787 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-osd (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:55.136 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-16T08:36:55.136 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-10-16T08:36:55.672 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-k8sevents (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:55.705 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-16T08:36:55.733 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-16T08:36:55.772 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T08:36:55.773 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-diskprediction-local (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:55.916 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mon (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:56.032 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-16T08:36:56.049 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-16T08:36:56.136 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-16T08:36:56.136 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-10-16T08:36:56.176 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T08:36:56.510 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-16T08:36:56.538 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-16T08:36:56.576 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T08:36:56.675 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-cephadm (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:56.827 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:56.921 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-16T08:36:56.949 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-10-16T08:36:56.953 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-mgr-dashboard (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:56.979 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-16T08:36:57.104 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ceph-volume (18.0.0-6675-gc7471068-1jammy) ... 2023-10-16T08:36:57.257 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-16T08:36:57.285 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-10-16T08:36:57.323 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-16T08:36:57.456 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-16T08:36:57.676 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libpq5:amd64. 2023-10-16T08:36:57.704 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:36:57.743 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:36:58.070 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-16T08:36:58.098 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:36:58.137 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T08:36:58.406 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-16T08:36:58.422 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:36:58.456 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T08:36:58.577 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-16T08:36:58.700 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-16T08:36:58.716 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.10_amd64.deb ... 2023-10-16T08:36:58.758 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-16T08:36:59.057 INFO:teuthology.orchestra.run.smithi040.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-16T08:36:59.060 DEBUG:teuthology.orchestra.run.smithi040:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-10-16T08:36:59.113 INFO:teuthology.orchestra.run.smithi040.stdout:Reading package lists... 2023-10-16T08:36:59.236 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-16T08:36:59.264 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-10-16T08:36:59.301 INFO:teuthology.orchestra.run.smithi040.stdout:Building dependency tree... 2023-10-16T08:36:59.302 INFO:teuthology.orchestra.run.smithi040.stdout:Reading state information... 2023-10-16T08:36:59.303 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-10-16T08:36:59.472 INFO:teuthology.orchestra.run.smithi040.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-10-16T08:36:59.472 INFO:teuthology.orchestra.run.smithi040.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-10-16T08:36:59.473 INFO:teuthology.orchestra.run.smithi040.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-10-16T08:36:59.473 INFO:teuthology.orchestra.run.smithi040.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-10-16T08:36:59.473 INFO:teuthology.orchestra.run.smithi040.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-10-16T08:36:59.473 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages were automatically installed and are no longer required: 2023-10-16T08:36:59.473 INFO:teuthology.orchestra.run.smithi040.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-10-16T08:36:59.473 INFO:teuthology.orchestra.run.smithi040.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-10-16T08:36:59.474 INFO:teuthology.orchestra.run.smithi040.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-10-16T08:36:59.474 INFO:teuthology.orchestra.run.smithi040.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-10-16T08:36:59.475 INFO:teuthology.orchestra.run.smithi040.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-10-16T08:36:59.475 INFO:teuthology.orchestra.run.smithi040.stdout:Use 'sudo apt autoremove' to remove them. 2023-10-16T08:36:59.475 INFO:teuthology.orchestra.run.smithi040.stdout:The following additional packages will be installed: 2023-10-16T08:36:59.476 INFO:teuthology.orchestra.run.smithi040.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-10-16T08:36:59.476 INFO:teuthology.orchestra.run.smithi040.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-10-16T08:36:59.476 INFO:teuthology.orchestra.run.smithi040.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-10-16T08:36:59.476 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-10-16T08:36:59.476 INFO:teuthology.orchestra.run.smithi040.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-16T08:36:59.477 INFO:teuthology.orchestra.run.smithi040.stdout:Suggested packages: 2023-10-16T08:36:59.477 INFO:teuthology.orchestra.run.smithi040.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-10-16T08:36:59.477 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-10-16T08:36:59.477 INFO:teuthology.orchestra.run.smithi040.stdout: libengine-pkcs11-openssl 2023-10-16T08:36:59.512 INFO:teuthology.orchestra.run.smithi040.stdout:The following NEW packages will be installed: 2023-10-16T08:36:59.513 INFO:teuthology.orchestra.run.smithi040.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-10-16T08:36:59.513 INFO:teuthology.orchestra.run.smithi040.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-10-16T08:36:59.514 INFO:teuthology.orchestra.run.smithi040.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-10-16T08:36:59.514 INFO:teuthology.orchestra.run.smithi040.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-10-16T08:36:59.514 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-10-16T08:36:59.514 INFO:teuthology.orchestra.run.smithi040.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-10-16T08:36:59.515 INFO:teuthology.orchestra.run.smithi040.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-10-16T08:36:59.515 INFO:teuthology.orchestra.run.smithi040.stdout:The following packages will be upgraded: 2023-10-16T08:36:59.515 INFO:teuthology.orchestra.run.smithi040.stdout: libssl3 2023-10-16T08:36:59.598 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package modemmanager. 2023-10-16T08:36:59.616 INFO:teuthology.orchestra.run.smithi040.stdout:1 upgraded, 39 newly installed, 0 to remove and 255 not upgraded. 2023-10-16T08:36:59.617 INFO:teuthology.orchestra.run.smithi040.stdout:Need to get 54.6 MB of archives. 2023-10-16T08:36:59.617 INFO:teuthology.orchestra.run.smithi040.stdout:After this operation, 202 MB of additional disk space will be used. 2023-10-16T08:36:59.617 INFO:teuthology.orchestra.run.smithi040.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-10-16T08:36:59.626 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-16T08:36:59.664 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T08:36:59.883 INFO:teuthology.orchestra.run.smithi040.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.10 [1,901 kB] 2023-10-16T08:37:00.077 INFO:teuthology.orchestra.run.smithi040.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-10-16T08:37:00.109 INFO:teuthology.orchestra.run.smithi040.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-10-16T08:37:00.110 INFO:teuthology.orchestra.run.smithi040.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-10-16T08:37:00.114 INFO:teuthology.orchestra.run.smithi040.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-10-16T08:37:00.116 INFO:teuthology.orchestra.run.smithi040.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-10-16T08:37:00.117 INFO:teuthology.orchestra.run.smithi040.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy/main amd64 libbluetooth3 amd64 5.64-0ubuntu1 [86.9 kB] 2023-10-16T08:37:00.120 INFO:teuthology.orchestra.run.smithi040.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-10-16T08:37:00.123 INFO:teuthology.orchestra.run.smithi040.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-10-16T08:37:00.123 INFO:teuthology.orchestra.run.smithi040.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-10-16T08:37:00.124 INFO:teuthology.orchestra.run.smithi040.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-10-16T08:37:00.156 INFO:teuthology.orchestra.run.smithi040.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-10-16T08:37:00.304 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package network-manager. 2023-10-16T08:37:00.332 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-10-16T08:37:00.378 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-10-16T08:37:00.808 INFO:teuthology.orchestra.run.smithi040.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-10-16T08:37:00.812 INFO:teuthology.orchestra.run.smithi040.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-10-16T08:37:00.813 INFO:teuthology.orchestra.run.smithi040.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-10-16T08:37:00.819 INFO:teuthology.orchestra.run.smithi040.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-10-16T08:37:00.820 INFO:teuthology.orchestra.run.smithi040.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-10-16T08:37:00.831 INFO:teuthology.orchestra.run.smithi040.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.9-0ubuntu0.22.04.1 [141 kB] 2023-10-16T08:37:00.834 INFO:teuthology.orchestra.run.smithi040.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-10-16T08:37:00.852 INFO:teuthology.orchestra.run.smithi040.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-10-16T08:37:00.852 INFO:teuthology.orchestra.run.smithi040.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.10 [2,372 kB] 2023-10-16T08:37:00.887 INFO:teuthology.orchestra.run.smithi040.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-10-16T08:37:00.888 INFO:teuthology.orchestra.run.smithi040.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-10-16T08:37:00.899 INFO:teuthology.orchestra.run.smithi040.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-10-16T08:37:00.941 INFO:teuthology.orchestra.run.smithi040.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-10-16T08:37:00.949 INFO:teuthology.orchestra.run.smithi040.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-10-16T08:37:00.950 INFO:teuthology.orchestra.run.smithi040.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-10-16T08:37:00.951 INFO:teuthology.orchestra.run.smithi040.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-10-16T08:37:00.952 INFO:teuthology.orchestra.run.smithi040.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.9-0ubuntu0.22.04.1 [1,222 kB] 2023-10-16T08:37:00.976 INFO:teuthology.orchestra.run.smithi040.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-10-16T08:37:00.978 INFO:teuthology.orchestra.run.smithi040.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.9-0ubuntu0.22.04.1 [16.1 MB] 2023-10-16T08:37:01.010 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package ppp. 2023-10-16T08:37:01.038 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-10-16T08:37:01.324 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-10-16T08:37:01.376 INFO:teuthology.orchestra.run.smithi040.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-10-16T08:37:01.376 INFO:teuthology.orchestra.run.smithi040.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-10-16T08:37:01.377 INFO:teuthology.orchestra.run.smithi040.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-10-16T08:37:01.377 INFO:teuthology.orchestra.run.smithi040.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-10-16T08:37:01.377 INFO:teuthology.orchestra.run.smithi040.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-10-16T08:37:01.402 INFO:teuthology.orchestra.run.smithi040.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-10-16T08:37:01.425 INFO:teuthology.orchestra.run.smithi040.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-10-16T08:37:01.426 INFO:teuthology.orchestra.run.smithi040.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-10-16T08:37:01.623 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package pptp-linux. 2023-10-16T08:37:01.650 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-10-16T08:37:01.689 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-10-16T08:37:01.851 INFO:teuthology.orchestra.run.smithi040.stdout:Preconfiguring packages ... 2023-10-16T08:37:01.942 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-16T08:37:01.969 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-10-16T08:37:02.008 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-10-16T08:37:02.012 INFO:teuthology.orchestra.run.smithi040.stdout:Fetched 54.6 MB in 2s (28.7 MB/s) 2023-10-16T08:37:02.184 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package flex. 2023-10-16T08:37:02.291 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172907 files and directories currently installed.) 2023-10-16T08:37:02.297 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-10-16T08:37:02.311 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-client-common. 2023-10-16T08:37:02.339 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-10-16T08:37:02.348 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking flex (2.6.4-8build2) ... 2023-10-16T08:37:02.378 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-client-common (238) ... 2023-10-16T08:37:02.640 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-client-14. 2023-10-16T08:37:02.667 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:37:02.706 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:37:02.941 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.10_amd64.deb ... 2023-10-16T08:37:03.113 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.10) over (3.0.2-0ubuntu1.7) ... 2023-10-16T08:37:03.294 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-common. 2023-10-16T08:37:03.322 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-10-16T08:37:03.379 INFO:teuthology.orchestra.run.smithi086.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-10-16T08:37:03.408 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-common (238) ... 2023-10-16T08:37:03.674 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-16T08:37:03.837 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-14. 2023-10-16T08:37:03.865 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:37:03.905 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:37:04.164 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package bison. 2023-10-16T08:37:04.229 INFO:teuthology.orchestra.run.smithi040.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 172980 files and directories currently installed.) 2023-10-16T08:37:04.235 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-10-16T08:37:04.267 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-10-16T08:37:04.693 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-10-16T08:37:04.721 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-10-16T08:37:04.782 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-10-16T08:37:05.130 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package dump. 2023-10-16T08:37:05.148 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql. 2023-10-16T08:37:05.157 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-10-16T08:37:05.164 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-10-16T08:37:05.196 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking dump (0.4b47-3) ... 2023-10-16T08:37:05.206 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql (14+238) ... 2023-10-16T08:37:05.458 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package indent. 2023-10-16T08:37:05.458 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-client. 2023-10-16T08:37:05.474 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:37:05.486 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-10-16T08:37:05.516 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-16T08:37:05.525 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-client (14+238) ... 2023-10-16T08:37:05.760 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package postgresql-contrib. 2023-10-16T08:37:05.777 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-10-16T08:37:05.777 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-10-16T08:37:05.805 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-10-16T08:37:05.810 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking postgresql-contrib (14+238) ... 2023-10-16T08:37:05.843 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-16T08:37:06.071 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package tcl8.6. 2023-10-16T08:37:06.099 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-10-16T08:37:06.137 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-16T08:37:06.172 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-10-16T08:37:06.200 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1_amd64.deb ... 2023-10-16T08:37:06.238 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-16T08:37:06.423 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package tcl. 2023-10-16T08:37:06.452 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-10-16T08:37:06.495 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-10-16T08:37:06.499 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-10-16T08:37:06.516 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-10-16T08:37:06.557 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-10-16T08:37:06.836 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libfl2:amd64. 2023-10-16T08:37:06.843 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package wpasupplicant. 2023-10-16T08:37:06.864 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-10-16T08:37:06.871 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-10-16T08:37:06.894 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-10-16T08:37:06.909 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-16T08:37:07.150 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-10-16T08:37:07.174 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-10-16T08:37:07.213 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-16T08:37:07.238 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-16T08:37:07.254 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-10-16T08:37:07.289 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-10-16T08:37:07.466 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libjson-perl. 2023-10-16T08:37:07.494 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-10-16T08:37:07.532 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-10-16T08:37:07.684 INFO:teuthology.orchestra.run.smithi086.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-16T08:37:07.711 INFO:teuthology.orchestra.run.smithi086.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-10-16T08:37:07.759 INFO:teuthology.orchestra.run.smithi086.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-16T08:37:07.852 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libllvm14:amd64. 2023-10-16T08:37:07.880 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-10-16T08:37:07.919 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-16T08:37:08.036 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-client-common (238) ... 2023-10-16T08:37:08.205 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up flex (2.6.4-8build2) ... 2023-10-16T08:37:08.332 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-16T08:37:08.458 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:37:08.584 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-10-16T08:37:08.710 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-16T08:37:08.836 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:37:09.348 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-10-16T08:37:09.449 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-16T08:37:09.589 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-10-16T08:37:09.617 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-16T08:37:09.655 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T08:37:09.828 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-16T08:37:09.828 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-16T08:37:09.899 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libmbim-proxy. 2023-10-16T08:37:09.916 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-10-16T08:37:09.958 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T08:37:10.277 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-10-16T08:37:10.306 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-16T08:37:10.343 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T08:37:10.377 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-10-16T08:37:10.696 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libndp0:amd64. 2023-10-16T08:37:10.724 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-10-16T08:37:10.763 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-16T08:37:10.954 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-16T08:37:11.041 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libnm0:amd64. 2023-10-16T08:37:11.068 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-10-16T08:37:11.095 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-10-16T08:37:11.099 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-16T08:37:11.238 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-10-16T08:37:11.339 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-10-16T08:37:11.452 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libpq5:amd64. 2023-10-16T08:37:11.456 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-16T08:37:11.481 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../16-libpq5_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:37:11.518 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:37:11.583 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-10-16T08:37:11.855 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-10-16T08:37:11.883 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:37:11.921 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T08:37:12.215 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libqmi-proxy. 2023-10-16T08:37:12.229 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-16T08:37:12.243 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:37:12.282 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T08:37:12.339 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up dump (0.4b47-3) ... 2023-10-16T08:37:12.416 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-16T08:37:12.475 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-10-16T08:37:12.482 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T08:37:12.492 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.10_amd64.deb ... 2023-10-16T08:37:12.534 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-16T08:37:12.608 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-16T08:37:12.885 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up tcl (8.6.11+1build2) ... 2023-10-16T08:37:13.010 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libjson-perl (4.04000-1) ... 2023-10-16T08:37:13.029 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-10-16T08:37:13.057 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-10-16T08:37:13.095 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-10-16T08:37:13.136 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-10-16T08:37:13.228 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-10-16T08:37:13.288 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-16T08:37:13.405 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-16T08:37:13.449 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package modemmanager. 2023-10-16T08:37:13.477 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-10-16T08:37:13.515 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T08:37:13.557 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-16T08:37:13.691 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T08:37:13.817 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-10-16T08:37:13.960 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-client (14+238) ... 2023-10-16T08:37:14.078 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-10-16T08:37:14.205 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package network-manager. 2023-10-16T08:37:14.234 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-10-16T08:37:14.280 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-10-16T08:37:14.575 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:14.575 INFO:teuthology.orchestra.run.smithi086.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-16T08:37:14.575 INFO:teuthology.orchestra.run.smithi086.stdout:which means they are currently configured by ifupdown: 2023-10-16T08:37:14.575 INFO:teuthology.orchestra.run.smithi086.stdout:- enp3s0f1 2023-10-16T08:37:14.576 INFO:teuthology.orchestra.run.smithi086.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-16T08:37:14.576 INFO:teuthology.orchestra.run.smithi086.stdout:remove their configuration from /etc/network/interfaces. 2023-10-16T08:37:14.576 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:14.688 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-16T08:37:14.837 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package ppp. 2023-10-16T08:37:14.865 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-10-16T08:37:15.077 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-16T08:37:15.155 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-10-16T08:37:15.433 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package pptp-linux. 2023-10-16T08:37:15.437 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-16T08:37:15.451 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-10-16T08:37:15.492 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-10-16T08:37:15.761 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package network-manager-pptp. 2023-10-16T08:37:15.789 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-10-16T08:37:15.828 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-10-16T08:37:15.981 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T08:37:16.133 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-16T08:37:16.164 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-client-common. 2023-10-16T08:37:16.192 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-10-16T08:37:16.230 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-client-common (238) ... 2023-10-16T08:37:16.258 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-10-16T08:37:16.384 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-common (238) ... 2023-10-16T08:37:16.501 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-client-14. 2023-10-16T08:37:16.517 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../27-postgresql-client-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:37:16.550 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:37:17.290 INFO:teuthology.orchestra.run.smithi086.stdout:Adding user postgres to group ssl-cert 2023-10-16T08:37:17.340 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-common. 2023-10-16T08:37:17.368 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-10-16T08:37:17.425 INFO:teuthology.orchestra.run.smithi040.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-10-16T08:37:17.454 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-common (238) ... 2023-10-16T08:37:17.527 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:17.527 INFO:teuthology.orchestra.run.smithi086.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-10-16T08:37:17.710 INFO:teuthology.orchestra.run.smithi086.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-10-16T08:37:17.711 INFO:teuthology.orchestra.run.smithi086.stdout:Removing obsolete dictionary files: 2023-10-16T08:37:17.905 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-14. 2023-10-16T08:37:17.933 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../29-postgresql-14_14.9-0ubuntu0.22.04.1_amd64.deb ... 2023-10-16T08:37:17.975 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:37:18.333 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-10-16T08:37:18.702 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:37:19.030 INFO:teuthology.orchestra.run.smithi086.stdout:Creating new PostgreSQL cluster 14/main ... 2023-10-16T08:37:19.046 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-10-16T08:37:19.088 INFO:teuthology.orchestra.run.smithi086.stdout:The files belonging to this database system will be owned by user "postgres". 2023-10-16T08:37:19.088 INFO:teuthology.orchestra.run.smithi086.stdout:This user must also own the server process. 2023-10-16T08:37:19.088 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:19.089 INFO:teuthology.orchestra.run.smithi086.stdout:The database cluster will be initialized with locale "en_US". 2023-10-16T08:37:19.089 INFO:teuthology.orchestra.run.smithi086.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-10-16T08:37:19.089 INFO:teuthology.orchestra.run.smithi086.stdout:The default text search configuration will be set to "english". 2023-10-16T08:37:19.089 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:19.089 INFO:teuthology.orchestra.run.smithi086.stdout:Data page checksums are disabled. 2023-10-16T08:37:19.090 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:19.090 INFO:teuthology.orchestra.run.smithi086.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-10-16T08:37:19.090 INFO:teuthology.orchestra.run.smithi086.stdout:creating subdirectories ... ok 2023-10-16T08:37:19.091 INFO:teuthology.orchestra.run.smithi086.stdout:selecting dynamic shared memory implementation ... posix 2023-10-16T08:37:19.101 INFO:teuthology.orchestra.run.smithi086.stdout:selecting default max_connections ... 100 2023-10-16T08:37:19.122 INFO:teuthology.orchestra.run.smithi086.stdout:selecting default shared_buffers ... 128MB 2023-10-16T08:37:19.132 INFO:teuthology.orchestra.run.smithi086.stdout:selecting default time zone ... Etc/UTC 2023-10-16T08:37:19.133 INFO:teuthology.orchestra.run.smithi086.stdout:creating configuration files ... ok 2023-10-16T08:37:19.246 INFO:teuthology.orchestra.run.smithi086.stdout:running bootstrap script ... ok 2023-10-16T08:37:19.843 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql. 2023-10-16T08:37:19.860 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-10-16T08:37:19.891 INFO:teuthology.orchestra.run.smithi086.stdout:performing post-bootstrap initialization ... ok 2023-10-16T08:37:19.901 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql (14+238) ... 2023-10-16T08:37:20.154 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-client. 2023-10-16T08:37:20.176 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-10-16T08:37:20.220 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-client (14+238) ... 2023-10-16T08:37:20.455 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package postgresql-contrib. 2023-10-16T08:37:20.472 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-10-16T08:37:20.514 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking postgresql-contrib (14+238) ... 2023-10-16T08:37:20.523 INFO:teuthology.orchestra.run.smithi086.stdout:syncing data to disk ... ok 2023-10-16T08:37:20.766 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package tcl8.6. 2023-10-16T08:37:20.795 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-10-16T08:37:20.833 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-16T08:37:20.926 INFO:teuthology.orchestra.run.smithi086.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-10-16T08:37:21.119 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package tcl. 2023-10-16T08:37:21.147 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-10-16T08:37:21.199 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-10-16T08:37:21.530 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package wpasupplicant. 2023-10-16T08:37:21.559 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-10-16T08:37:21.596 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-16T08:37:21.960 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-10-16T08:37:21.976 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-10-16T08:37:22.009 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-10-16T08:37:22.379 INFO:teuthology.orchestra.run.smithi040.stdout:Selecting previously unselected package usb-modeswitch. 2023-10-16T08:37:22.408 INFO:teuthology.orchestra.run.smithi040.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-10-16T08:37:22.454 INFO:teuthology.orchestra.run.smithi040.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-16T08:37:22.766 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-client-common (238) ... 2023-10-16T08:37:22.952 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up flex (2.6.4-8build2) ... 2023-10-16T08:37:23.078 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-10-16T08:37:23.204 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libpq5:amd64 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:37:23.330 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-10-16T08:37:23.456 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-10-16T08:37:23.582 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-client-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:37:23.759 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T08:37:23.885 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql-contrib (14+238) ... 2023-10-16T08:37:24.011 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up postgresql (14+238) ... 2023-10-16T08:37:24.105 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-10-16T08:37:24.137 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T08:37:24.188 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-10-16T08:37:24.255 INFO:teuthology.orchestra.run.smithi086.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T08:37:24.492 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-16T08:37:24.492 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-16T08:37:24.566 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-16T08:37:24.566 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-10-16T08:37:25.057 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-16T08:37:25.132 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-10-16T08:37:25.233 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.10) ... 2023-10-16T08:37:25.334 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-10-16T08:37:25.368 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-16T08:37:25.443 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-10-16T08:37:25.541 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-16T08:37:25.569 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-10-16T08:37:25.695 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-10-16T08:37:25.821 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-10-16T08:37:26.493 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-10-16T08:37:26.728 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up dump (0.4b47-3) ... 2023-10-16T08:37:26.805 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-10-16T08:37:26.862 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T08:37:26.863 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-10-16T08:37:26.963 INFO:teuthology.orchestra.run.smithi086.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-10-16T08:37:26.972 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1) ... 2023-10-16T08:37:27.098 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up tcl (8.6.11+1build2) ... 2023-10-16T08:37:27.224 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libjson-perl (4.04000-1) ... 2023-10-16T08:37:27.350 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-10-16T08:37:27.424 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-10-16T08:37:27.485 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-10-16T08:37:27.599 INFO:teuthology.orchestra.run.smithi086.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-16T08:37:27.602 DEBUG:teuthology.parallel:result is None 2023-10-16T08:37:27.610 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-10-16T08:37:27.796 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-10-16T08:37:27.921 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T08:37:28.047 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-10-16T08:37:28.215 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-client (14+238) ... 2023-10-16T08:37:28.325 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-10-16T08:37:28.761 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:28.761 INFO:teuthology.orchestra.run.smithi040.stdout:The following network interfaces were found in /etc/network/interfaces 2023-10-16T08:37:28.761 INFO:teuthology.orchestra.run.smithi040.stdout:which means they are currently configured by ifupdown: 2023-10-16T08:37:28.761 INFO:teuthology.orchestra.run.smithi040.stdout:- ens1f0 2023-10-16T08:37:28.762 INFO:teuthology.orchestra.run.smithi040.stdout:If you want to manage those interfaces with NetworkManager instead 2023-10-16T08:37:28.762 INFO:teuthology.orchestra.run.smithi040.stdout:remove their configuration from /etc/network/interfaces. 2023-10-16T08:37:28.762 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:28.892 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-10-16T08:37:29.217 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-10-16T08:37:29.559 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-10-16T08:37:30.186 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-10-16T08:37:30.329 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-10-16T08:37:30.455 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-10-16T08:37:30.581 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-common (238) ... 2023-10-16T08:37:31.427 INFO:teuthology.orchestra.run.smithi040.stdout:Adding user postgres to group ssl-cert 2023-10-16T08:37:31.665 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:31.665 INFO:teuthology.orchestra.run.smithi040.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-10-16T08:37:31.857 INFO:teuthology.orchestra.run.smithi040.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-10-16T08:37:31.858 INFO:teuthology.orchestra.run.smithi040.stdout:Removing obsolete dictionary files: 2023-10-16T08:37:32.453 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-10-16T08:37:32.841 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-14 (14.9-0ubuntu0.22.04.1) ... 2023-10-16T08:37:33.167 INFO:teuthology.orchestra.run.smithi040.stdout:Creating new PostgreSQL cluster 14/main ... 2023-10-16T08:37:33.181 INFO:teuthology.orchestra.run.smithi040.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-10-16T08:37:33.223 INFO:teuthology.orchestra.run.smithi040.stdout:The files belonging to this database system will be owned by user "postgres". 2023-10-16T08:37:33.224 INFO:teuthology.orchestra.run.smithi040.stdout:This user must also own the server process. 2023-10-16T08:37:33.224 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:33.224 INFO:teuthology.orchestra.run.smithi040.stdout:The database cluster will be initialized with locale "en_US". 2023-10-16T08:37:33.224 INFO:teuthology.orchestra.run.smithi040.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-10-16T08:37:33.225 INFO:teuthology.orchestra.run.smithi040.stdout:The default text search configuration will be set to "english". 2023-10-16T08:37:33.225 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:33.225 INFO:teuthology.orchestra.run.smithi040.stdout:Data page checksums are disabled. 2023-10-16T08:37:33.225 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:33.225 INFO:teuthology.orchestra.run.smithi040.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-10-16T08:37:33.225 INFO:teuthology.orchestra.run.smithi040.stdout:creating subdirectories ... ok 2023-10-16T08:37:33.226 INFO:teuthology.orchestra.run.smithi040.stdout:selecting dynamic shared memory implementation ... posix 2023-10-16T08:37:33.240 INFO:teuthology.orchestra.run.smithi040.stdout:selecting default max_connections ... 100 2023-10-16T08:37:33.258 INFO:teuthology.orchestra.run.smithi040.stdout:selecting default shared_buffers ... 128MB 2023-10-16T08:37:33.265 INFO:teuthology.orchestra.run.smithi040.stdout:selecting default time zone ... Etc/UTC 2023-10-16T08:37:33.266 INFO:teuthology.orchestra.run.smithi040.stdout:creating configuration files ... ok 2023-10-16T08:37:33.377 INFO:teuthology.orchestra.run.smithi040.stdout:running bootstrap script ... ok 2023-10-16T08:37:34.066 INFO:teuthology.orchestra.run.smithi040.stdout:performing post-bootstrap initialization ... ok 2023-10-16T08:37:34.634 INFO:teuthology.orchestra.run.smithi040.stdout:syncing data to disk ... ok 2023-10-16T08:37:35.081 INFO:teuthology.orchestra.run.smithi040.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-10-16T08:37:37.881 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T08:37:38.006 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql-contrib (14+238) ... 2023-10-16T08:37:38.133 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up postgresql (14+238) ... 2023-10-16T08:37:38.258 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-10-16T08:37:38.384 INFO:teuthology.orchestra.run.smithi040.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-10-16T08:37:38.628 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-10-16T08:37:38.629 INFO:teuthology.orchestra.run.smithi040.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-10-16T08:37:39.237 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-10-16T08:37:39.548 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-10-16T08:37:39.691 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-10-16T08:37:41.035 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-10-16T08:37:41.135 INFO:teuthology.orchestra.run.smithi040.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-10-16T08:37:41.809 INFO:teuthology.orchestra.run.smithi040.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-10-16T08:37:41.812 DEBUG:teuthology.parallel:result is None 2023-10-16T08:37:41.813 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c7471068426370a3754c2250e7729280233d552b 2023-10-16T08:37:41.952 DEBUG:teuthology.orchestra.run.smithi040:> dpkg-query -W -f '${Version}' ceph 2023-10-16T08:37:41.972 INFO:teuthology.orchestra.run.smithi040.stdout:18.0.0-6675-gc7471068-1jammy 2023-10-16T08:37:41.973 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6675-gc7471068-1jammy 2023-10-16T08:37:41.973 INFO:teuthology.task.install:The correct ceph version 18.0.0-6675-gc7471068-1jammy is installed. 2023-10-16T08:37:41.973 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=c7471068426370a3754c2250e7729280233d552b 2023-10-16T08:37:42.220 DEBUG:teuthology.orchestra.run.smithi086:> dpkg-query -W -f '${Version}' ceph 2023-10-16T08:37:42.239 INFO:teuthology.orchestra.run.smithi086.stdout:18.0.0-6675-gc7471068-1jammy 2023-10-16T08:37:42.240 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-6675-gc7471068-1jammy 2023-10-16T08:37:42.240 INFO:teuthology.task.install:The correct ceph version 18.0.0-6675-gc7471068-1jammy is installed. 2023-10-16T08:37:42.240 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-16T08:37:42.241 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:37:42.241 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-16T08:37:42.256 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:37:42.257 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-16T08:37:42.294 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-16T08:37:42.294 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:37:42.294 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/daemon-helper 2023-10-16T08:37:42.310 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-16T08:37:42.366 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:37:42.367 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2023-10-16T08:37:42.379 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-16T08:37:42.433 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-16T08:37:42.434 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:37:42.434 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-16T08:37:42.448 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-16T08:37:42.505 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:37:42.506 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-16T08:37:42.519 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-16T08:37:42.572 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-16T08:37:42.573 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:37:42.573 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/usr/bin/stdin-killer 2023-10-16T08:37:42.587 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-16T08:37:42.645 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:37:42.646 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2023-10-16T08:37:42.659 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-16T08:37:42.713 INFO:teuthology.run_tasks:Running task ceph... 2023-10-16T08:37:42.772 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-16T08:37:42.773 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 777 /var/log/ceph 2023-10-16T08:37:42.776 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /var/log/ceph 2023-10-16T08:37:42.789 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-16T08:37:42.789 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-16T08:37:42.831 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-16T08:37:42.844 INFO:tasks.ceph:Creating extra log directories... 2023-10-16T08:37:42.844 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-16T08:37:42.892 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-16T08:37:42.911 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-16T08:37:42.912 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-16T08:37:42.912 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-16T08:37:42.912 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:37:42.913 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-16T08:37:42.951 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-16T08:37:43.027 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:37:43.027 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-16T08:37:43.033 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-16T08:37:43.115 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-16T08:37:43.115 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-16T08:37:43.115 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/mchangir-2023-10-16_08:10:12-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428951', 'branch': 'wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': 'mchangir@redhat.com', 'first_in_suite': False, 'job_id': '7428951', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'mchangir-2023-10-16_08:10:12-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'c7471068426370a3754c2250e7729280233d552b'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'c7471068426370a3754c2250e7729280233d552b'}}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'mgr-snap_schedule-support-subvol-and-group-arguments', 'sha1': '6d9e81bfe885e1cd43cce3f41c9616ed0dce2cf3'}}, 'owner': 'mchangir', 'priority': 25, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 748, 'sha1': 'c7471068426370a3754c2250e7729280233d552b', 'sleep_before_teardown': 0, 'suite': 'fs', 'suite_branch': 'mgr-snap_schedule-support-subvol-and-group-arguments', 'suite_path': '/home/teuthworker/src/github.com_mchangir_ceph_6d9e81bfe885e1cd43cce3f41c9616ed0dce2cf3/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/mchangir/ceph.git', 'suite_sha1': '6d9e81bfe885e1cd43cce3f41c9616ed0dce2cf3', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi040.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC9NQL/MGgwiIOjSeKTp24cQ97ZwOsJMk9LstleBGMYYh52MDJnTBeLIx4/B8MBdtdmz/qKA4NUnTfa70peh9nk=', 'smithi086.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIPx1U1nPl6AeNRotZqILZpt6k01E0TmhKhCzRMRxWKyrwm7FoM4l+t0BUkfIsIT2DsTx0DwhEW/OSe+eW4DzRo='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'c7471068426370a3754c2250e7729280233d552b'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': 'c7471068426370a3754c2250e7729280233d552b', 'cluster': 'ceph'}}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '8cdab074dcca9a68965bc5a50e9c30b691949723', 'timestamp': '2023-10-16_08:10:12', 'tube': 'smithi', 'user': 'mchangir', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-16T08:37:43.116 DEBUG:teuthology.orchestra.run.smithi040:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-16T08:37:43.119 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-16T08:37:43.158 DEBUG:teuthology.orchestra.run.smithi040:> sudo install -d -m0777 -- /var/run/ceph 2023-10-16T08:37:43.168 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m0777 -- /var/run/ceph 2023-10-16T08:37:43.214 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:37:43.214 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/scratch_devs of=/dev/stdout 2023-10-16T08:37:43.227 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-16T08:37:43.227 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_1 2023-10-16T08:37:43.275 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-16T08:37:43.275 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T08:37:43.275 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 884 Links: 1 2023-10-16T08:37:43.276 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T08:37:43.276 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-10-16 08:31:32.217772063 +0000 2023-10-16T08:37:43.276 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-10-16 08:31:31.845775386 +0000 2023-10-16T08:37:43.276 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-10-16 08:31:31.845775386 +0000 2023-10-16T08:37:43.276 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2023-10-16T08:37:43.277 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-16T08:37:43.329 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2023-10-16T08:37:43.329 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2023-10-16T08:37:43.329 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000336285 s, 1.5 MB/s 2023-10-16T08:37:43.330 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-16T08:37:43.380 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_2 2023-10-16T08:37:43.426 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-16T08:37:43.427 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T08:37:43.427 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 894 Links: 1 2023-10-16T08:37:43.427 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T08:37:43.427 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-10-16 08:31:32.849766418 +0000 2023-10-16T08:37:43.427 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-10-16 08:31:32.469769812 +0000 2023-10-16T08:37:43.428 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-10-16 08:31:32.469769812 +0000 2023-10-16T08:37:43.428 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2023-10-16T08:37:43.428 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-16T08:37:43.481 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2023-10-16T08:37:43.482 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2023-10-16T08:37:43.482 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000370274 s, 1.4 MB/s 2023-10-16T08:37:43.483 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-16T08:37:43.532 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_3 2023-10-16T08:37:43.578 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-16T08:37:43.579 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T08:37:43.579 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 912 Links: 1 2023-10-16T08:37:43.579 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T08:37:43.579 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-10-16 08:31:33.453761024 +0000 2023-10-16T08:37:43.580 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-10-16 08:31:33.093764239 +0000 2023-10-16T08:37:43.580 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-10-16 08:31:33.093764239 +0000 2023-10-16T08:37:43.580 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2023-10-16T08:37:43.580 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-16T08:37:43.633 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2023-10-16T08:37:43.633 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2023-10-16T08:37:43.633 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000367325 s, 1.4 MB/s 2023-10-16T08:37:43.634 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-16T08:37:43.684 DEBUG:teuthology.orchestra.run.smithi040:> stat /dev/vg_nvme/lv_4 2023-10-16T08:37:43.730 INFO:teuthology.orchestra.run.smithi040.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-16T08:37:43.731 INFO:teuthology.orchestra.run.smithi040.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T08:37:43.731 INFO:teuthology.orchestra.run.smithi040.stdout:Device: 5h/5d Inode: 935 Links: 1 2023-10-16T08:37:43.731 INFO:teuthology.orchestra.run.smithi040.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T08:37:43.731 INFO:teuthology.orchestra.run.smithi040.stdout:Access: 2023-10-16 08:31:34.057755628 +0000 2023-10-16T08:37:43.731 INFO:teuthology.orchestra.run.smithi040.stdout:Modify: 2023-10-16 08:31:33.681758987 +0000 2023-10-16T08:37:43.732 INFO:teuthology.orchestra.run.smithi040.stdout:Change: 2023-10-16 08:31:33.681758987 +0000 2023-10-16T08:37:43.732 INFO:teuthology.orchestra.run.smithi040.stdout: Birth: - 2023-10-16T08:37:43.732 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-16T08:37:43.785 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records in 2023-10-16T08:37:43.785 INFO:teuthology.orchestra.run.smithi040.stderr:1+0 records out 2023-10-16T08:37:43.786 INFO:teuthology.orchestra.run.smithi040.stderr:512 bytes copied, 0.000364014 s, 1.4 MB/s 2023-10-16T08:37:43.787 DEBUG:teuthology.orchestra.run.smithi040:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-16T08:37:43.836 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T08:37:43.836 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:37:43.836 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2023-10-16T08:37:43.842 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-16T08:37:43.843 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2023-10-16T08:37:43.890 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-16T08:37:43.890 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T08:37:43.891 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 880 Links: 1 2023-10-16T08:37:43.891 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T08:37:43.891 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2023-10-16 08:31:31.797254436 +0000 2023-10-16T08:37:43.891 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2023-10-16 08:31:31.793254472 +0000 2023-10-16T08:37:43.891 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2023-10-16 08:31:31.793254472 +0000 2023-10-16T08:37:43.892 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2023-10-16T08:37:43.892 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-16T08:37:43.944 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2023-10-16T08:37:43.944 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2023-10-16T08:37:43.945 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000383212 s, 1.3 MB/s 2023-10-16T08:37:43.946 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-16T08:37:43.995 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2023-10-16T08:37:44.042 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-16T08:37:44.042 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T08:37:44.042 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 897 Links: 1 2023-10-16T08:37:44.042 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T08:37:44.043 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2023-10-16 08:31:32.773245719 +0000 2023-10-16T08:37:44.043 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2023-10-16 08:31:32.409248969 +0000 2023-10-16T08:37:44.043 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2023-10-16 08:31:32.409248969 +0000 2023-10-16T08:37:44.043 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2023-10-16T08:37:44.044 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-16T08:37:44.096 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2023-10-16T08:37:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2023-10-16T08:37:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.00038321 s, 1.3 MB/s 2023-10-16T08:37:44.098 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-16T08:37:44.147 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2023-10-16T08:37:44.194 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-16T08:37:44.194 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T08:37:44.195 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 919 Links: 1 2023-10-16T08:37:44.195 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T08:37:44.195 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2023-10-16 08:31:33.361240466 +0000 2023-10-16T08:37:44.195 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2023-10-16 08:31:33.005243645 +0000 2023-10-16T08:37:44.195 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2023-10-16 08:31:33.005243645 +0000 2023-10-16T08:37:44.196 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2023-10-16T08:37:44.196 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-16T08:37:44.248 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2023-10-16T08:37:44.248 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2023-10-16T08:37:44.248 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000402032 s, 1.3 MB/s 2023-10-16T08:37:44.249 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-16T08:37:44.299 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2023-10-16T08:37:44.350 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-16T08:37:44.350 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-16T08:37:44.350 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 931 Links: 1 2023-10-16T08:37:44.351 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-16T08:37:44.351 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2023-10-16 08:31:33.961235107 +0000 2023-10-16T08:37:44.351 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2023-10-16 08:31:33.605238287 +0000 2023-10-16T08:37:44.351 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2023-10-16 08:31:33.605238287 +0000 2023-10-16T08:37:44.351 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: - 2023-10-16T08:37:44.352 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-16T08:37:44.404 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2023-10-16T08:37:44.404 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2023-10-16T08:37:44.405 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000411159 s, 1.2 MB/s 2023-10-16T08:37:44.406 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-16T08:37:44.455 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T08:37:44.456 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi086.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-16T08:37:44.456 INFO:tasks.ceph:Generating config... 2023-10-16T08:37:44.457 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-16T08:37:44.458 INFO:tasks.ceph:[client] debug client = 20 2023-10-16T08:37:44.458 INFO:tasks.ceph:[client] debug ms = 1 2023-10-16T08:37:44.458 INFO:tasks.ceph:[client] pid file = 2023-10-16T08:37:44.459 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-16T08:37:44.459 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-16T08:37:44.459 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-16T08:37:44.459 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-10-16T08:37:44.460 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-16T08:37:44.460 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-16T08:37:44.460 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-16T08:37:44.461 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-16T08:37:44.461 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-16T08:37:44.461 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-16T08:37:44.461 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-16T08:37:44.462 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-16T08:37:44.462 INFO:tasks.ceph:[mgr] debug client = 20 2023-10-16T08:37:44.462 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-10-16T08:37:44.463 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-16T08:37:44.463 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-16T08:37:44.463 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-16T08:37:44.463 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-16T08:37:44.464 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-16T08:37:44.464 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-16T08:37:44.464 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-16T08:37:44.464 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-16T08:37:44.465 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-16T08:37:44.465 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-16T08:37:44.465 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-16T08:37:44.465 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-16T08:37:44.466 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-16T08:37:44.466 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-16T08:37:44.466 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-16T08:37:44.466 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-16T08:37:44.467 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-16T08:37:44.467 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-16T08:37:44.467 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-16T08:37:44.468 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-16T08:37:44.468 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-16T08:37:44.468 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-16T08:37:44.468 INFO:tasks.ceph:Setting up mon.a... 2023-10-16T08:37:44.469 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-16T08:37:44.503 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.keyring 2023-10-16T08:37:44.512 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-16T08:37:44.591 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-16T08:37:44.651 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.40'), ('mon.b', '172.21.15.86'), ('mon.c', '[v2:172.21.15.86:3301,v1:172.21.15.86:6790]')] 2023-10-16T08:37:44.651 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.40,172.21.15.86,[v2:172.21.15.86:3301,v1:172.21.15.86:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-16T08:37:44.652 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:37:44.653 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-16T08:37:44.699 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.40 --add b 172.21.15.86 --addv c '[v2:172.21.15.86:3301,v1:172.21.15.86:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:37:44.785 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:37:44.786 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: generated fsid cc6c59f7-0179-4f54-b71d-89fa67221543 2023-10-16T08:37:44.786 INFO:teuthology.orchestra.run.smithi040.stdout:setting min_mon_release = pacific 2023-10-16T08:37:44.786 INFO:teuthology.orchestra.run.smithi040.stdout:epoch 0 2023-10-16T08:37:44.786 INFO:teuthology.orchestra.run.smithi040.stdout:fsid cc6c59f7-0179-4f54-b71d-89fa67221543 2023-10-16T08:37:44.787 INFO:teuthology.orchestra.run.smithi040.stdout:last_changed 2023-10-16T08:37:44.786489+0000 2023-10-16T08:37:44.787 INFO:teuthology.orchestra.run.smithi040.stdout:created 2023-10-16T08:37:44.786489+0000 2023-10-16T08:37:44.787 INFO:teuthology.orchestra.run.smithi040.stdout:min_mon_release 16 (pacific) 2023-10-16T08:37:44.787 INFO:teuthology.orchestra.run.smithi040.stdout:election_strategy: 1 2023-10-16T08:37:44.787 INFO:teuthology.orchestra.run.smithi040.stdout:0: [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] mon.a 2023-10-16T08:37:44.788 INFO:teuthology.orchestra.run.smithi040.stdout:1: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.b 2023-10-16T08:37:44.788 INFO:teuthology.orchestra.run.smithi040.stdout:2: [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] mon.c 2023-10-16T08:37:44.788 INFO:teuthology.orchestra.run.smithi040.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-16T08:37:44.789 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-16T08:37:44.835 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID cc6c59f7-0179-4f54-b71d-89fa67221543... 2023-10-16T08:37:44.836 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-16T08:37:44.879 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-16T08:37:44.910 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2023-10-16T08:37:44.910 INFO:teuthology.orchestra.run.smithi086.stdout: chdir = "" 2023-10-16T08:37:44.911 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-16T08:37:44.911 INFO:teuthology.orchestra.run.smithi086.stdout: auth supported = cephx 2023-10-16T08:37:44.911 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.911 INFO:teuthology.orchestra.run.smithi086.stdout: filestore xattr use omap = true 2023-10-16T08:37:44.911 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.911 INFO:teuthology.orchestra.run.smithi086.stdout: mon clock drift allowed = 1.000 2023-10-16T08:37:44.912 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.912 INFO:teuthology.orchestra.run.smithi086.stdout: osd crush chooseleaf type = 0 2023-10-16T08:37:44.912 INFO:teuthology.orchestra.run.smithi086.stdout: auth debug = true 2023-10-16T08:37:44.912 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.912 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on old message = true 2023-10-16T08:37:44.913 INFO:teuthology.orchestra.run.smithi086.stdout: ms die on bug = true 2023-10-16T08:37:44.913 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.913 INFO:teuthology.orchestra.run.smithi086.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-16T08:37:44.913 INFO:teuthology.orchestra.run.smithi086.stdout: mon pg warn max object skew = 0 2023-10-16T08:37:44.913 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.913 INFO:teuthology.orchestra.run.smithi086.stdout: # disable pg_autoscaler by default for new pools 2023-10-16T08:37:44.914 INFO:teuthology.orchestra.run.smithi086.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-16T08:37:44.914 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.914 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default size = 2 2023-10-16T08:37:44.914 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.914 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow primary affinity = true 2023-10-16T08:37:44.915 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd allow pg remap = true 2023-10-16T08:37:44.915 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on legacy crush tunables = false 2023-10-16T08:37:44.915 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on crush straw calc version zero = false 2023-10-16T08:37:44.915 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on no sortbitwise = false 2023-10-16T08:37:44.915 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on osd down out interval zero = false 2023-10-16T08:37:44.915 INFO:teuthology.orchestra.run.smithi086.stdout: mon warn on too few osds = false 2023-10-16T08:37:44.916 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-16T08:37:44.916 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-16T08:37:44.916 INFO:teuthology.orchestra.run.smithi086.stdout: mon_allow_pool_size_one = true 2023-10-16T08:37:44.916 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.916 INFO:teuthology.orchestra.run.smithi086.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-16T08:37:44.916 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.917 INFO:teuthology.orchestra.run.smithi086.stdout: osd default data pool replay window = 5 2023-10-16T08:37:44.917 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.917 INFO:teuthology.orchestra.run.smithi086.stdout: mon allow pool delete = true 2023-10-16T08:37:44.917 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.918 INFO:teuthology.orchestra.run.smithi086.stdout: mon cluster log file level = debug 2023-10-16T08:37:44.918 INFO:teuthology.orchestra.run.smithi086.stdout: debug asserts on shutdown = true 2023-10-16T08:37:44.918 INFO:teuthology.orchestra.run.smithi086.stdout: mon health detail to clog = false 2023-10-16T08:37:44.918 INFO:teuthology.orchestra.run.smithi086.stdout: mon host = "172.21.15.40,172.21.15.86,[v2:172.21.15.86:3301,v1:172.21.15.86:6790]" 2023-10-16T08:37:44.918 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = cc6c59f7-0179-4f54-b71d-89fa67221543 2023-10-16T08:37:44.918 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.919 INFO:teuthology.orchestra.run.smithi086.stdout:[osd] 2023-10-16T08:37:44.919 INFO:teuthology.orchestra.run.smithi086.stdout: osd journal size = 100 2023-10-16T08:37:44.919 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.919 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub load threshold = 5.0 2023-10-16T08:37:44.919 INFO:teuthology.orchestra.run.smithi086.stdout: osd scrub max interval = 600 2023-10-16T08:37:44.919 INFO:teuthology.orchestra.run.smithi086.stdout: osd mclock profile = high_recovery_ops 2023-10-16T08:37:44.920 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.920 INFO:teuthology.orchestra.run.smithi086.stdout: osd recover clone overlap = true 2023-10-16T08:37:44.920 INFO:teuthology.orchestra.run.smithi086.stdout: osd recovery max chunk = 1048576 2023-10-16T08:37:44.920 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.920 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug shutdown = true 2023-10-16T08:37:44.921 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug op order = true 2023-10-16T08:37:44.921 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug verify stray on activate = true 2023-10-16T08:37:44.921 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.921 INFO:teuthology.orchestra.run.smithi086.stdout: osd open classes on start = true 2023-10-16T08:37:44.921 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug pg log writeout = true 2023-10-16T08:37:44.921 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.922 INFO:teuthology.orchestra.run.smithi086.stdout: osd deep scrub update digest min age = 30 2023-10-16T08:37:44.922 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.922 INFO:teuthology.orchestra.run.smithi086.stdout: osd map max advance = 10 2023-10-16T08:37:44.922 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.922 INFO:teuthology.orchestra.run.smithi086.stdout: journal zero on create = true 2023-10-16T08:37:44.923 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.923 INFO:teuthology.orchestra.run.smithi086.stdout: filestore ondisk finisher threads = 3 2023-10-16T08:37:44.923 INFO:teuthology.orchestra.run.smithi086.stdout: filestore apply finisher threads = 3 2023-10-16T08:37:44.923 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.923 INFO:teuthology.orchestra.run.smithi086.stdout: bdev debug aio = true 2023-10-16T08:37:44.923 INFO:teuthology.orchestra.run.smithi086.stdout: osd debug misdirected ops = true 2023-10-16T08:37:44.924 INFO:teuthology.orchestra.run.smithi086.stdout: bdev async discard = True 2023-10-16T08:37:44.924 INFO:teuthology.orchestra.run.smithi086.stdout: bdev enable discard = True 2023-10-16T08:37:44.924 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore allocator = bitmap 2023-10-16T08:37:44.924 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore block size = 96636764160 2023-10-16T08:37:44.925 INFO:teuthology.orchestra.run.smithi086.stdout: bluestore fsck on mount = True 2023-10-16T08:37:44.925 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluefs = 1/20 2023-10-16T08:37:44.925 INFO:teuthology.orchestra.run.smithi086.stdout: debug bluestore = 1/20 2023-10-16T08:37:44.925 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2023-10-16T08:37:44.925 INFO:teuthology.orchestra.run.smithi086.stdout: debug osd = 20 2023-10-16T08:37:44.926 INFO:teuthology.orchestra.run.smithi086.stdout: debug rocksdb = 4/10 2023-10-16T08:37:44.926 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd backfillfull_ratio = 0.85 2023-10-16T08:37:44.926 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd full ratio = 0.9 2023-10-16T08:37:44.926 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd nearfull ratio = 0.8 2023-10-16T08:37:44.926 INFO:teuthology.orchestra.run.smithi086.stdout: osd failsafe full ratio = 0.95 2023-10-16T08:37:44.927 INFO:teuthology.orchestra.run.smithi086.stdout: osd objectstore = bluestore 2023-10-16T08:37:44.927 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2023-10-16T08:37:44.927 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.927 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr] 2023-10-16T08:37:44.928 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2023-10-16T08:37:44.928 INFO:teuthology.orchestra.run.smithi086.stdout: debug mgr = 20 2023-10-16T08:37:44.928 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2023-10-16T08:37:44.928 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2023-10-16T08:37:44.928 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2023-10-16T08:37:44.928 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2023-10-16T08:37:44.929 INFO:teuthology.orchestra.run.smithi086.stdout: mgr/telemetry/nag = false 2023-10-16T08:37:44.929 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2023-10-16T08:37:44.929 INFO:teuthology.orchestra.run.smithi086.stdout: debug finisher = 20 2023-10-16T08:37:44.929 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.929 INFO:teuthology.orchestra.run.smithi086.stdout:[mon] 2023-10-16T08:37:44.930 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2023-10-16T08:37:44.930 INFO:teuthology.orchestra.run.smithi086.stdout: debug mon = 20 2023-10-16T08:37:44.930 INFO:teuthology.orchestra.run.smithi086.stdout: debug paxos = 20 2023-10-16T08:37:44.930 INFO:teuthology.orchestra.run.smithi086.stdout: debug auth = 20 2023-10-16T08:37:44.930 INFO:teuthology.orchestra.run.smithi086.stdout: mon data avail warn = 5 2023-10-16T08:37:44.931 INFO:teuthology.orchestra.run.smithi086.stdout: mon mgr mkfs grace = 240 2023-10-16T08:37:44.931 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min pgs per osd = 4 2023-10-16T08:37:44.931 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd reporter subtree level = osd 2023-10-16T08:37:44.931 INFO:teuthology.orchestra.run.smithi086.stdout: mon osd prime pg temp = true 2023-10-16T08:37:44.931 INFO:teuthology.orchestra.run.smithi086.stdout: mon reweight min bytes per osd = 10 2023-10-16T08:37:44.932 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.932 INFO:teuthology.orchestra.run.smithi086.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-16T08:37:44.932 INFO:teuthology.orchestra.run.smithi086.stdout: auth mon ticket ttl = 660 # 11m 2023-10-16T08:37:44.932 INFO:teuthology.orchestra.run.smithi086.stdout: auth service ticket ttl = 240 # 4m 2023-10-16T08:37:44.932 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.933 INFO:teuthology.orchestra.run.smithi086.stdout: # don't complain about insecure global_id in the test suite 2023-10-16T08:37:44.933 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-16T08:37:44.933 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-16T08:37:44.933 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.933 INFO:teuthology.orchestra.run.smithi086.stdout: # 1m isn't quite enough 2023-10-16T08:37:44.934 INFO:teuthology.orchestra.run.smithi086.stdout: mon_down_mkfs_grace = 2m 2023-10-16T08:37:44.934 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.934 INFO:teuthology.orchestra.run.smithi086.stdout: mon_warn_on_filestore_osds = false 2023-10-16T08:37:44.934 INFO:teuthology.orchestra.run.smithi086.stdout: mon op complaint time = 120 2023-10-16T08:37:44.934 INFO:teuthology.orchestra.run.smithi086.stdout: 2023-10-16T08:37:44.935 INFO:teuthology.orchestra.run.smithi086.stdout:[client] 2023-10-16T08:37:44.935 INFO:teuthology.orchestra.run.smithi086.stdout: rgw cache enabled = true 2023-10-16T08:37:44.935 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable ops log = true 2023-10-16T08:37:44.935 INFO:teuthology.orchestra.run.smithi086.stdout: rgw enable usage log = true 2023-10-16T08:37:44.935 INFO:teuthology.orchestra.run.smithi086.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-16T08:37:44.935 INFO:teuthology.orchestra.run.smithi086.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-16T08:37:44.936 INFO:teuthology.orchestra.run.smithi086.stdout: client mount timeout = 600 2023-10-16T08:37:44.936 INFO:teuthology.orchestra.run.smithi086.stdout: debug client = 20 2023-10-16T08:37:44.936 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2023-10-16T08:37:44.936 INFO:teuthology.orchestra.run.smithi086.stdout: pid file = "" 2023-10-16T08:37:44.936 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2023-10-16T08:37:44.936 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2023-10-16T08:37:44.937 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.a] 2023-10-16T08:37:44.937 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.b] 2023-10-16T08:37:44.937 INFO:teuthology.orchestra.run.smithi086.stdout:[mon.c] 2023-10-16T08:37:44.937 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2023-10-16T08:37:44.937 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2023-10-16T08:37:44.937 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2023-10-16T08:37:44.938 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2023-10-16T08:37:44.938 INFO:teuthology.orchestra.run.smithi086.stdout:[mds] 2023-10-16T08:37:44.938 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds = 20 2023-10-16T08:37:44.938 INFO:teuthology.orchestra.run.smithi086.stdout: debug mds balancer = 20 2023-10-16T08:37:44.938 INFO:teuthology.orchestra.run.smithi086.stdout: debug ms = 1 2023-10-16T08:37:44.938 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug frag = True 2023-10-16T08:37:44.939 INFO:teuthology.orchestra.run.smithi086.stdout: mds debug scatterstat = True 2023-10-16T08:37:44.939 INFO:teuthology.orchestra.run.smithi086.stdout: mds op complaint time = 180 2023-10-16T08:37:44.939 INFO:teuthology.orchestra.run.smithi086.stdout: mds verify scatter = True 2023-10-16T08:37:44.939 INFO:teuthology.orchestra.run.smithi086.stdout: osd op complaint time = 180 2023-10-16T08:37:44.939 INFO:teuthology.orchestra.run.smithi086.stdout: rados mon op timeout = 900 2023-10-16T08:37:44.940 INFO:teuthology.orchestra.run.smithi086.stdout: rados osd op timeout = 900 2023-10-16T08:37:44.941 INFO:teuthology.orchestra.run.smithi040.stdout:[global] 2023-10-16T08:37:44.941 INFO:teuthology.orchestra.run.smithi040.stdout: chdir = "" 2023-10-16T08:37:44.942 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-16T08:37:44.942 INFO:teuthology.orchestra.run.smithi040.stdout: auth supported = cephx 2023-10-16T08:37:44.942 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.942 INFO:teuthology.orchestra.run.smithi040.stdout: filestore xattr use omap = true 2023-10-16T08:37:44.942 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.943 INFO:teuthology.orchestra.run.smithi040.stdout: mon clock drift allowed = 1.000 2023-10-16T08:37:44.943 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.943 INFO:teuthology.orchestra.run.smithi040.stdout: osd crush chooseleaf type = 0 2023-10-16T08:37:44.943 INFO:teuthology.orchestra.run.smithi040.stdout: auth debug = true 2023-10-16T08:37:44.943 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.944 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on old message = true 2023-10-16T08:37:44.944 INFO:teuthology.orchestra.run.smithi040.stdout: ms die on bug = true 2023-10-16T08:37:44.944 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.944 INFO:teuthology.orchestra.run.smithi040.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-16T08:37:44.944 INFO:teuthology.orchestra.run.smithi040.stdout: mon pg warn max object skew = 0 2023-10-16T08:37:44.945 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.945 INFO:teuthology.orchestra.run.smithi040.stdout: # disable pg_autoscaler by default for new pools 2023-10-16T08:37:44.945 INFO:teuthology.orchestra.run.smithi040.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-16T08:37:44.945 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.945 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default size = 2 2023-10-16T08:37:44.946 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.946 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow primary affinity = true 2023-10-16T08:37:44.946 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd allow pg remap = true 2023-10-16T08:37:44.946 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on legacy crush tunables = false 2023-10-16T08:37:44.946 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on crush straw calc version zero = false 2023-10-16T08:37:44.947 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on no sortbitwise = false 2023-10-16T08:37:44.947 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on osd down out interval zero = false 2023-10-16T08:37:44.947 INFO:teuthology.orchestra.run.smithi040.stdout: mon warn on too few osds = false 2023-10-16T08:37:44.947 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-16T08:37:44.947 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-16T08:37:44.948 INFO:teuthology.orchestra.run.smithi040.stdout: mon_allow_pool_size_one = true 2023-10-16T08:37:44.948 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.948 INFO:teuthology.orchestra.run.smithi040.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-16T08:37:44.948 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.949 INFO:teuthology.orchestra.run.smithi040.stdout: osd default data pool replay window = 5 2023-10-16T08:37:44.949 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.949 INFO:teuthology.orchestra.run.smithi040.stdout: mon allow pool delete = true 2023-10-16T08:37:44.949 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.949 INFO:teuthology.orchestra.run.smithi040.stdout: mon cluster log file level = debug 2023-10-16T08:37:44.950 INFO:teuthology.orchestra.run.smithi040.stdout: debug asserts on shutdown = true 2023-10-16T08:37:44.950 INFO:teuthology.orchestra.run.smithi040.stdout: mon health detail to clog = false 2023-10-16T08:37:44.950 INFO:teuthology.orchestra.run.smithi040.stdout: mon host = "172.21.15.40,172.21.15.86,[v2:172.21.15.86:3301,v1:172.21.15.86:6790]" 2023-10-16T08:37:44.950 INFO:teuthology.orchestra.run.smithi040.stdout: fsid = cc6c59f7-0179-4f54-b71d-89fa67221543 2023-10-16T08:37:44.950 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.951 INFO:teuthology.orchestra.run.smithi040.stdout:[osd] 2023-10-16T08:37:44.951 INFO:teuthology.orchestra.run.smithi040.stdout: osd journal size = 100 2023-10-16T08:37:44.951 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.951 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub load threshold = 5.0 2023-10-16T08:37:44.951 INFO:teuthology.orchestra.run.smithi040.stdout: osd scrub max interval = 600 2023-10-16T08:37:44.951 INFO:teuthology.orchestra.run.smithi040.stdout: osd mclock profile = high_recovery_ops 2023-10-16T08:37:44.952 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.952 INFO:teuthology.orchestra.run.smithi040.stdout: osd recover clone overlap = true 2023-10-16T08:37:44.952 INFO:teuthology.orchestra.run.smithi040.stdout: osd recovery max chunk = 1048576 2023-10-16T08:37:44.952 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.952 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug shutdown = true 2023-10-16T08:37:44.953 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug op order = true 2023-10-16T08:37:44.953 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug verify stray on activate = true 2023-10-16T08:37:44.953 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.953 INFO:teuthology.orchestra.run.smithi040.stdout: osd open classes on start = true 2023-10-16T08:37:44.953 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug pg log writeout = true 2023-10-16T08:37:44.953 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.954 INFO:teuthology.orchestra.run.smithi040.stdout: osd deep scrub update digest min age = 30 2023-10-16T08:37:44.954 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.954 INFO:teuthology.orchestra.run.smithi040.stdout: osd map max advance = 10 2023-10-16T08:37:44.954 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.954 INFO:teuthology.orchestra.run.smithi040.stdout: journal zero on create = true 2023-10-16T08:37:44.955 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.955 INFO:teuthology.orchestra.run.smithi040.stdout: filestore ondisk finisher threads = 3 2023-10-16T08:37:44.955 INFO:teuthology.orchestra.run.smithi040.stdout: filestore apply finisher threads = 3 2023-10-16T08:37:44.955 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.955 INFO:teuthology.orchestra.run.smithi040.stdout: bdev debug aio = true 2023-10-16T08:37:44.956 INFO:teuthology.orchestra.run.smithi040.stdout: osd debug misdirected ops = true 2023-10-16T08:37:44.956 INFO:teuthology.orchestra.run.smithi040.stdout: bdev async discard = True 2023-10-16T08:37:44.956 INFO:teuthology.orchestra.run.smithi040.stdout: bdev enable discard = True 2023-10-16T08:37:44.956 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore allocator = bitmap 2023-10-16T08:37:44.956 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore block size = 96636764160 2023-10-16T08:37:44.957 INFO:teuthology.orchestra.run.smithi040.stdout: bluestore fsck on mount = True 2023-10-16T08:37:44.957 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluefs = 1/20 2023-10-16T08:37:44.957 INFO:teuthology.orchestra.run.smithi040.stdout: debug bluestore = 1/20 2023-10-16T08:37:44.957 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2023-10-16T08:37:44.957 INFO:teuthology.orchestra.run.smithi040.stdout: debug osd = 20 2023-10-16T08:37:44.958 INFO:teuthology.orchestra.run.smithi040.stdout: debug rocksdb = 4/10 2023-10-16T08:37:44.958 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd backfillfull_ratio = 0.85 2023-10-16T08:37:44.958 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd full ratio = 0.9 2023-10-16T08:37:44.958 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd nearfull ratio = 0.8 2023-10-16T08:37:44.958 INFO:teuthology.orchestra.run.smithi040.stdout: osd failsafe full ratio = 0.95 2023-10-16T08:37:44.959 INFO:teuthology.orchestra.run.smithi040.stdout: osd objectstore = bluestore 2023-10-16T08:37:44.959 INFO:teuthology.orchestra.run.smithi040.stdout: osd op complaint time = 180 2023-10-16T08:37:44.959 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.959 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr] 2023-10-16T08:37:44.959 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2023-10-16T08:37:44.960 INFO:teuthology.orchestra.run.smithi040.stdout: debug mgr = 20 2023-10-16T08:37:44.960 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2023-10-16T08:37:44.960 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2023-10-16T08:37:44.960 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2023-10-16T08:37:44.960 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2023-10-16T08:37:44.960 INFO:teuthology.orchestra.run.smithi040.stdout: mgr/telemetry/nag = false 2023-10-16T08:37:44.961 INFO:teuthology.orchestra.run.smithi040.stdout: debug client = 20 2023-10-16T08:37:44.961 INFO:teuthology.orchestra.run.smithi040.stdout: debug finisher = 20 2023-10-16T08:37:44.961 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.961 INFO:teuthology.orchestra.run.smithi040.stdout:[mon] 2023-10-16T08:37:44.961 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2023-10-16T08:37:44.962 INFO:teuthology.orchestra.run.smithi040.stdout: debug mon = 20 2023-10-16T08:37:44.962 INFO:teuthology.orchestra.run.smithi040.stdout: debug paxos = 20 2023-10-16T08:37:44.962 INFO:teuthology.orchestra.run.smithi040.stdout: debug auth = 20 2023-10-16T08:37:44.962 INFO:teuthology.orchestra.run.smithi040.stdout: mon data avail warn = 5 2023-10-16T08:37:44.962 INFO:teuthology.orchestra.run.smithi040.stdout: mon mgr mkfs grace = 240 2023-10-16T08:37:44.963 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min pgs per osd = 4 2023-10-16T08:37:44.963 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd reporter subtree level = osd 2023-10-16T08:37:44.963 INFO:teuthology.orchestra.run.smithi040.stdout: mon osd prime pg temp = true 2023-10-16T08:37:44.963 INFO:teuthology.orchestra.run.smithi040.stdout: mon reweight min bytes per osd = 10 2023-10-16T08:37:44.963 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.963 INFO:teuthology.orchestra.run.smithi040.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-16T08:37:44.964 INFO:teuthology.orchestra.run.smithi040.stdout: auth mon ticket ttl = 660 # 11m 2023-10-16T08:37:44.964 INFO:teuthology.orchestra.run.smithi040.stdout: auth service ticket ttl = 240 # 4m 2023-10-16T08:37:44.964 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.964 INFO:teuthology.orchestra.run.smithi040.stdout: # don't complain about insecure global_id in the test suite 2023-10-16T08:37:44.964 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-16T08:37:44.965 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-16T08:37:44.965 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.965 INFO:teuthology.orchestra.run.smithi040.stdout: # 1m isn't quite enough 2023-10-16T08:37:44.965 INFO:teuthology.orchestra.run.smithi040.stdout: mon_down_mkfs_grace = 2m 2023-10-16T08:37:44.965 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.965 INFO:teuthology.orchestra.run.smithi040.stdout: mon_warn_on_filestore_osds = false 2023-10-16T08:37:44.966 INFO:teuthology.orchestra.run.smithi040.stdout: mon op complaint time = 120 2023-10-16T08:37:44.966 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:37:44.966 INFO:teuthology.orchestra.run.smithi040.stdout:[client] 2023-10-16T08:37:44.966 INFO:teuthology.orchestra.run.smithi040.stdout: rgw cache enabled = true 2023-10-16T08:37:44.966 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable ops log = true 2023-10-16T08:37:44.967 INFO:teuthology.orchestra.run.smithi040.stdout: rgw enable usage log = true 2023-10-16T08:37:44.967 INFO:teuthology.orchestra.run.smithi040.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-16T08:37:44.967 INFO:teuthology.orchestra.run.smithi040.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-16T08:37:44.967 INFO:teuthology.orchestra.run.smithi040.stdout: client mount timeout = 600 2023-10-16T08:37:44.967 INFO:teuthology.orchestra.run.smithi040.stdout: debug client = 20 2023-10-16T08:37:44.968 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2023-10-16T08:37:44.968 INFO:teuthology.orchestra.run.smithi040.stdout: pid file = "" 2023-10-16T08:37:44.968 INFO:teuthology.orchestra.run.smithi040.stdout: rados mon op timeout = 900 2023-10-16T08:37:44.968 INFO:teuthology.orchestra.run.smithi040.stdout: rados osd op timeout = 900 2023-10-16T08:37:44.968 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.a] 2023-10-16T08:37:44.968 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.b] 2023-10-16T08:37:44.969 INFO:teuthology.orchestra.run.smithi040.stdout:[mon.c] 2023-10-16T08:37:44.969 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a] 2023-10-16T08:37:44.969 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.c] 2023-10-16T08:37:44.969 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b] 2023-10-16T08:37:44.969 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.d] 2023-10-16T08:37:44.970 INFO:teuthology.orchestra.run.smithi040.stdout:[mds] 2023-10-16T08:37:44.970 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds = 20 2023-10-16T08:37:44.970 INFO:teuthology.orchestra.run.smithi040.stdout: debug mds balancer = 20 2023-10-16T08:37:44.970 INFO:teuthology.orchestra.run.smithi040.stdout: debug ms = 1 2023-10-16T08:37:44.970 INFO:teuthology.orchestra.run.smithi040.stdout: mds debug frag = True 2023-10-16T08:37:44.971 INFO:teuthology.orchestra.run.smithi040.stdout: mds debug scatterstat = True 2023-10-16T08:37:44.971 INFO:teuthology.orchestra.run.smithi040.stdout: mds op complaint time = 180 2023-10-16T08:37:44.971 INFO:teuthology.orchestra.run.smithi040.stdout: mds verify scatter = True 2023-10-16T08:37:44.971 INFO:teuthology.orchestra.run.smithi040.stdout: osd op complaint time = 180 2023-10-16T08:37:44.971 INFO:teuthology.orchestra.run.smithi040.stdout: rados mon op timeout = 900 2023-10-16T08:37:44.972 INFO:teuthology.orchestra.run.smithi040.stdout: rados osd op timeout = 900 2023-10-16T08:37:44.972 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-16T08:37:44.972 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-16T08:37:45.033 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-16T08:37:45.033 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:37:45.034 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-16T08:37:45.039 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:37:45.040 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-16T08:37:45.087 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi040.front.sepia.ceph.com 2023-10-16T08:37:45.087 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:37:45.087 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-16T08:37:45.088 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-16T08:37:45.152 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:37:45.152 DEBUG:teuthology.orchestra.run.smithi040:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:37:45.199 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi086.front.sepia.ceph.com 2023-10-16T08:37:45.200 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:37:45.200 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-16T08:37:45.200 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-16T08:37:45.224 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:37:45.224 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:37:45.270 INFO:tasks.ceph:Setting up mon nodes... 2023-10-16T08:37:45.270 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-16T08:37:45.271 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-16T08:37:45.316 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-16T08:37:45.319 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-16T08:37:45.362 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-16T08:37:45.378 INFO:tasks.ceph:Setting up mds nodes... 2023-10-16T08:37:45.378 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-16T08:37:45.419 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-16T08:37:45.422 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-16T08:37:45.478 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-16T08:37:45.563 INFO:teuthology.orchestra.run.smithi040.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-16T08:37:45.566 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-16T08:37:45.622 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-16T08:37:45.666 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-16T08:37:45.669 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-16T08:37:45.724 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-16T08:37:45.805 INFO:teuthology.orchestra.run.smithi086.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-16T08:37:45.808 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-16T08:37:45.861 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-16T08:37:45.861 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-16T08:37:45.900 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-16T08:37:45.911 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-16T08:37:45.991 INFO:teuthology.orchestra.run.smithi040.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-16T08:37:46.002 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-16T08:37:46.038 INFO:teuthology.orchestra.run.smithi086.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-16T08:37:46.049 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-16T08:37:46.124 INFO:teuthology.orchestra.run.smithi086.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-16T08:37:46.134 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-16T08:37:46.134 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi040.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi086.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-16T08:37:46.135 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-16T08:37:46.149 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T08:37:46.149 INFO:tasks.ceph:role: osd.0 2023-10-16T08:37:46.150 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com 2023-10-16T08:37:46.150 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-16T08:37:46.211 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T08:37:46.212 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T08:37:46.212 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T08:37:46.212 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T08:37:46.212 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T08:37:46.212 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2023-10-16T08:37:46.213 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T08:37:46.213 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T08:37:46.213 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T08:37:46.213 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T08:37:46.666 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2023-10-16T08:37:46.668 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2023-10-16T08:37:46.669 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-16T08:37:46.748 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-16T08:37:46.799 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:37:46.800 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T08:37:46.801 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-16T08:37:46.853 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T08:37:46.853 INFO:tasks.ceph:role: osd.1 2023-10-16T08:37:46.853 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com 2023-10-16T08:37:46.854 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-16T08:37:46.915 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T08:37:46.915 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T08:37:46.916 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T08:37:46.916 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T08:37:46.916 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T08:37:46.916 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2023-10-16T08:37:46.916 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T08:37:46.917 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T08:37:46.917 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T08:37:46.917 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T08:37:47.398 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2023-10-16T08:37:47.400 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2023-10-16T08:37:47.400 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-16T08:37:47.546 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-16T08:37:47.558 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T08:37:47.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:37:47.559 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-16T08:37:47.614 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T08:37:47.615 INFO:tasks.ceph:role: osd.2 2023-10-16T08:37:47.615 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com 2023-10-16T08:37:47.615 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-16T08:37:47.678 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T08:37:47.678 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T08:37:47.679 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T08:37:47.679 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T08:37:47.679 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T08:37:47.679 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2023-10-16T08:37:47.679 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T08:37:47.680 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T08:37:47.680 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T08:37:47.680 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T08:37:48.157 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2023-10-16T08:37:48.159 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2023-10-16T08:37:48.159 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-16T08:37:48.308 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-16T08:37:48.320 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T08:37:48.321 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:37:48.321 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-16T08:37:48.374 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-16T08:37:48.374 INFO:tasks.ceph:role: osd.3 2023-10-16T08:37:48.375 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com 2023-10-16T08:37:48.375 DEBUG:teuthology.orchestra.run.smithi040:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-16T08:37:48.438 INFO:teuthology.orchestra.run.smithi040.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T08:37:48.438 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T08:37:48.438 INFO:teuthology.orchestra.run.smithi040.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T08:37:48.439 INFO:teuthology.orchestra.run.smithi040.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T08:37:48.439 INFO:teuthology.orchestra.run.smithi040.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T08:37:48.439 INFO:teuthology.orchestra.run.smithi040.stdout: = sunit=0 swidth=0 blks 2023-10-16T08:37:48.439 INFO:teuthology.orchestra.run.smithi040.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T08:37:48.439 INFO:teuthology.orchestra.run.smithi040.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T08:37:48.440 INFO:teuthology.orchestra.run.smithi040.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T08:37:48.440 INFO:teuthology.orchestra.run.smithi040.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T08:37:48.879 INFO:teuthology.orchestra.run.smithi040.stdout:Discarding blocks...Done. 2023-10-16T08:37:48.882 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi040.front.sepia.ceph.com -o noatime 2023-10-16T08:37:48.882 DEBUG:teuthology.orchestra.run.smithi040:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-16T08:37:48.974 DEBUG:teuthology.orchestra.run.smithi040:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-16T08:37:48.986 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:37:48.986 INFO:teuthology.orchestra.run.smithi040.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T08:37:48.987 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:37:49.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:49.097+0000 7fccb10e1640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-16T08:37:49.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:49.097+0000 7fccb10e1640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-16T08:37:49.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:49.097+0000 7fccb10e1640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-16T08:37:49.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:49.097+0000 7fccb10e1640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-16T08:37:49.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:49.097+0000 7fccb10e1640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-16T08:37:49.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:49.105+0000 7fccb10e1640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-16T08:37:52.108 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-16T08:37:52.122 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:37:52.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:52.205+0000 7f6d7f377640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-16T08:37:52.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:52.205+0000 7f6d7f377640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-16T08:37:52.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:52.205+0000 7f6d7f377640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-16T08:37:52.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:52.205+0000 7f6d7f377640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-16T08:37:52.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:52.205+0000 7f6d7f377640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-16T08:37:52.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:52.205+0000 7f6d7f377640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-16T08:37:55.240 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-16T08:37:55.299 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:37:55.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:55.381+0000 7f22e2d8f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-16T08:37:55.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:55.381+0000 7f22e2d8f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-16T08:37:55.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:55.381+0000 7f22e2d8f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-16T08:37:55.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:55.381+0000 7f22e2d8f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-16T08:37:55.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:55.381+0000 7f22e2d8f640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-16T08:37:55.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:55.381+0000 7f22e2d8f640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-16T08:37:58.398 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-16T08:37:58.456 DEBUG:teuthology.orchestra.run.smithi040:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:37:58.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:58.537+0000 7ff95dcb8640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-16T08:37:58.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:58.537+0000 7ff95dcb8640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-16T08:37:58.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:58.537+0000 7ff95dcb8640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-16T08:37:58.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:58.537+0000 7ff95dcb8640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-16T08:37:58.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:58.537+0000 7ff95dcb8640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-16T08:37:58.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:37:58.537+0000 7ff95dcb8640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-16T08:38:01.555 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-16T08:38:01.611 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-16T08:38:01.626 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T08:38:01.627 INFO:tasks.ceph:role: osd.4 2023-10-16T08:38:01.627 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi086.front.sepia.ceph.com 2023-10-16T08:38:01.627 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-16T08:38:01.689 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T08:38:01.689 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T08:38:01.689 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T08:38:01.690 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T08:38:01.690 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T08:38:01.690 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2023-10-16T08:38:01.690 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T08:38:01.690 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T08:38:01.691 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T08:38:01.691 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T08:38:02.155 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2023-10-16T08:38:02.157 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2023-10-16T08:38:02.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-16T08:38:02.270 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-16T08:38:02.282 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:38:02.282 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T08:38:02.283 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-16T08:38:02.336 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T08:38:02.337 INFO:tasks.ceph:role: osd.5 2023-10-16T08:38:02.337 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi086.front.sepia.ceph.com 2023-10-16T08:38:02.337 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-16T08:38:02.401 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T08:38:02.401 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T08:38:02.402 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T08:38:02.402 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T08:38:02.402 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T08:38:02.402 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2023-10-16T08:38:02.402 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T08:38:02.402 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T08:38:02.403 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T08:38:02.403 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T08:38:02.875 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2023-10-16T08:38:02.877 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2023-10-16T08:38:02.878 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-16T08:38:03.018 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-16T08:38:03.030 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:38:03.030 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T08:38:03.031 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-16T08:38:03.085 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T08:38:03.086 INFO:tasks.ceph:role: osd.6 2023-10-16T08:38:03.086 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi086.front.sepia.ceph.com 2023-10-16T08:38:03.086 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-16T08:38:03.149 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T08:38:03.150 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T08:38:03.150 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T08:38:03.150 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T08:38:03.150 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T08:38:03.151 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2023-10-16T08:38:03.151 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T08:38:03.151 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T08:38:03.151 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T08:38:03.151 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T08:38:03.627 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2023-10-16T08:38:03.630 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2023-10-16T08:38:03.630 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-16T08:38:03.774 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-16T08:38:03.785 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T08:38:03.785 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:38:03.786 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-16T08:38:03.840 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-16T08:38:03.841 INFO:tasks.ceph:role: osd.7 2023-10-16T08:38:03.841 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi086.front.sepia.ceph.com 2023-10-16T08:38:03.841 DEBUG:teuthology.orchestra.run.smithi086:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-16T08:38:03.904 INFO:teuthology.orchestra.run.smithi086.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-16T08:38:03.905 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-16T08:38:03.905 INFO:teuthology.orchestra.run.smithi086.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-16T08:38:03.905 INFO:teuthology.orchestra.run.smithi086.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-16T08:38:03.905 INFO:teuthology.orchestra.run.smithi086.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-16T08:38:03.905 INFO:teuthology.orchestra.run.smithi086.stdout: = sunit=0 swidth=0 blks 2023-10-16T08:38:03.906 INFO:teuthology.orchestra.run.smithi086.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-16T08:38:03.906 INFO:teuthology.orchestra.run.smithi086.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-16T08:38:03.906 INFO:teuthology.orchestra.run.smithi086.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-16T08:38:03.906 INFO:teuthology.orchestra.run.smithi086.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-16T08:38:04.334 INFO:teuthology.orchestra.run.smithi086.stdout:Discarding blocks...Done. 2023-10-16T08:38:04.336 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi086.front.sepia.ceph.com -o noatime 2023-10-16T08:38:04.336 DEBUG:teuthology.orchestra.run.smithi086:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-16T08:38:04.436 DEBUG:teuthology.orchestra.run.smithi086:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-16T08:38:04.447 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: /sbin/restorecon: command not found 2023-10-16T08:38:04.448 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:38:04.448 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:38:04.559 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:04.558+0000 7fe6a9ec6640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-16T08:38:04.562 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:04.562+0000 7fe6a9ec6640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-16T08:38:04.563 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:04.562+0000 7fe6a9ec6640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-16T08:38:04.563 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:04.562+0000 7fe6a9ec6640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-16T08:38:04.563 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:04.562+0000 7fe6a9ec6640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-16T08:38:04.566 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:04.566+0000 7fe6a9ec6640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-16T08:38:07.579 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-16T08:38:07.592 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:38:07.681 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:07.682+0000 7f906c3bc640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-16T08:38:07.682 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:07.682+0000 7f906c3bc640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-16T08:38:07.682 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:07.682+0000 7f906c3bc640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-16T08:38:07.682 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:07.682+0000 7f906c3bc640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-16T08:38:07.683 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:07.682+0000 7f906c3bc640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-16T08:38:07.683 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:07.682+0000 7f906c3bc640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-16T08:38:10.685 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-16T08:38:10.742 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:38:10.824 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:10.822+0000 7f71ebbee640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-16T08:38:10.825 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:10.826+0000 7f71ebbee640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-16T08:38:10.825 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:10.826+0000 7f71ebbee640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-16T08:38:10.825 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:10.826+0000 7f71ebbee640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-16T08:38:10.826 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:10.826+0000 7f71ebbee640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-16T08:38:10.826 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:10.826+0000 7f71ebbee640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-16T08:38:13.117 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:38:13.120 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:38:13.857 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-16T08:38:13.871 DEBUG:teuthology.orchestra.run.smithi086:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:38:13.953 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:13.954+0000 7fdae47ba640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-16T08:38:13.954 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:13.954+0000 7fdae47ba640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-16T08:38:13.954 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:13.954+0000 7fdae47ba640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-16T08:38:13.954 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:13.954+0000 7fdae47ba640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-16T08:38:13.954 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:13.954+0000 7fdae47ba640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-16T08:38:13.955 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:13.954+0000 7fdae47ba640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-16T08:38:16.972 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-16T08:38:17.031 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-16T08:38:17.031 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:38:17.031 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-16T08:38:17.046 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:38:17.047 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-16T08:38:17.102 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:38:17.102 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-16T08:38:17.158 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:38:17.158 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-16T08:38:17.214 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:38:17.215 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-16T08:38:17.270 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:38:17.270 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-16T08:38:17.326 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:38:17.326 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-16T08:38:17.382 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:38:17.382 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-16T08:38:17.395 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:38:17.395 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-16T08:38:17.448 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:38:17.448 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-16T08:38:17.501 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:38:17.501 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-16T08:38:17.557 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:38:17.557 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-16T08:38:17.612 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:38:17.612 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-16T08:38:17.664 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:38:17.664 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-16T08:38:17.716 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:38:17.716 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-16T08:38:17.722 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:38:17.723 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-16T08:38:17.771 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:38:17.771 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-16T08:38:17.776 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:38:17.777 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-16T08:38:17.825 INFO:tasks.ceph:Adding keys to all mons... 2023-10-16T08:38:17.825 DEBUG:teuthology.orchestra.run.smithi040:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-16T08:38:17.828 DEBUG:teuthology.orchestra.run.smithi086:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-16T08:38:17.871 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.y] 2023-10-16T08:38:17.871 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBZ9ixl+kLkEhAActTuQtt0Jrwq/iiowMCa9w== 2023-10-16T08:38:17.872 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.a] 2023-10-16T08:38:17.872 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBZ9ixl3zsLGRAAd1G0EsVzdx7C4F0E//FVFQ== 2023-10-16T08:38:17.872 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.c] 2023-10-16T08:38:17.872 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBZ9ixlJNOgIRAAnJ1DCdMUWzG4FGzwKFRJWg== 2023-10-16T08:38:17.872 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.0] 2023-10-16T08:38:17.873 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBd9ixl5XL6BRAAjRLriu66VCpXHsRraeIZYA== 2023-10-16T08:38:17.873 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.1] 2023-10-16T08:38:17.873 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBg9ixlQ8xsDBAAC3SfyGcs5LDUHFezkaSe0A== 2023-10-16T08:38:17.873 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.2] 2023-10-16T08:38:17.873 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBj9ixlSnjlFhAAkegxIZpQ8FGaxjmEVVz4jA== 2023-10-16T08:38:17.874 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.3] 2023-10-16T08:38:17.874 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBm9ixl8hoxIBAAychM4ZEXMkGuz9en6dVz+w== 2023-10-16T08:38:17.874 INFO:teuthology.orchestra.run.smithi040.stdout:[mgr.x] 2023-10-16T08:38:17.874 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBZ9ixlAPa8FRAAF9HAweeE9iSy6EOdyiEYCQ== 2023-10-16T08:38:17.874 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.b] 2023-10-16T08:38:17.874 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBZ9ixlxirfJxAAzzp97COKjSApTHnCFZCYvw== 2023-10-16T08:38:17.875 INFO:teuthology.orchestra.run.smithi040.stdout:[mds.d] 2023-10-16T08:38:17.875 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBZ9ixl5xUhMBAAde/8WUrGURn+SJvgfxv78w== 2023-10-16T08:38:17.875 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.4] 2023-10-16T08:38:17.875 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBs9ixl4/t4IRAAn4Y8Qj1Pe8y4py5fekDrdg== 2023-10-16T08:38:17.875 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.5] 2023-10-16T08:38:17.876 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBv9ixllPrCKBAAzioN5CEGwlw1hPiTGJ1j2A== 2023-10-16T08:38:17.876 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.6] 2023-10-16T08:38:17.876 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBy9ixl4tJOMRAAMWU7w6MwVATwOlm0tg61ow== 2023-10-16T08:38:17.876 INFO:teuthology.orchestra.run.smithi040.stdout:[osd.7] 2023-10-16T08:38:17.876 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQB19ixlyQj5OBAA6B/LtRfGS4w3CO0YNBWHnw== 2023-10-16T08:38:17.877 INFO:teuthology.orchestra.run.smithi040.stdout:[client.0] 2023-10-16T08:38:17.877 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBZ9ixluTq2NRAAu2d5FD9TEktWTvIi1OyVVg== 2023-10-16T08:38:17.877 INFO:teuthology.orchestra.run.smithi040.stdout:[client.2] 2023-10-16T08:38:17.877 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBZ9ixlHRolOxAAG/9KoJvTBAi4BNql0+I3xw== 2023-10-16T08:38:17.877 INFO:teuthology.orchestra.run.smithi040.stdout:[client.1] 2023-10-16T08:38:17.877 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBa9ixlcwFqAhAArt0fq+fo8Jny91S02+ewLA== 2023-10-16T08:38:17.878 INFO:teuthology.orchestra.run.smithi040.stdout:[client.3] 2023-10-16T08:38:17.878 INFO:teuthology.orchestra.run.smithi040.stdout: key = AQBa9ixlgmKSBxAAw9uL06dASc+t66uiLzdaUw== 2023-10-16T08:38:17.879 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr.y] 2023-10-16T08:38:17.880 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBZ9ixl+kLkEhAActTuQtt0Jrwq/iiowMCa9w== 2023-10-16T08:38:17.880 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a] 2023-10-16T08:38:17.880 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBZ9ixl3zsLGRAAd1G0EsVzdx7C4F0E//FVFQ== 2023-10-16T08:38:17.880 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c] 2023-10-16T08:38:17.880 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBZ9ixlJNOgIRAAnJ1DCdMUWzG4FGzwKFRJWg== 2023-10-16T08:38:17.881 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.0] 2023-10-16T08:38:17.881 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBd9ixl5XL6BRAAjRLriu66VCpXHsRraeIZYA== 2023-10-16T08:38:17.881 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.1] 2023-10-16T08:38:17.881 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBg9ixlQ8xsDBAAC3SfyGcs5LDUHFezkaSe0A== 2023-10-16T08:38:17.881 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.2] 2023-10-16T08:38:17.881 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBj9ixlSnjlFhAAkegxIZpQ8FGaxjmEVVz4jA== 2023-10-16T08:38:17.882 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.3] 2023-10-16T08:38:17.882 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBm9ixl8hoxIBAAychM4ZEXMkGuz9en6dVz+w== 2023-10-16T08:38:17.882 INFO:teuthology.orchestra.run.smithi086.stdout:[mgr.x] 2023-10-16T08:38:17.882 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBZ9ixlAPa8FRAAF9HAweeE9iSy6EOdyiEYCQ== 2023-10-16T08:38:17.882 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b] 2023-10-16T08:38:17.883 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBZ9ixlxirfJxAAzzp97COKjSApTHnCFZCYvw== 2023-10-16T08:38:17.883 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d] 2023-10-16T08:38:17.883 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBZ9ixl5xUhMBAAde/8WUrGURn+SJvgfxv78w== 2023-10-16T08:38:17.883 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.4] 2023-10-16T08:38:17.883 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBs9ixl4/t4IRAAn4Y8Qj1Pe8y4py5fekDrdg== 2023-10-16T08:38:17.883 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.5] 2023-10-16T08:38:17.884 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBv9ixllPrCKBAAzioN5CEGwlw1hPiTGJ1j2A== 2023-10-16T08:38:17.884 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.6] 2023-10-16T08:38:17.884 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBy9ixl4tJOMRAAMWU7w6MwVATwOlm0tg61ow== 2023-10-16T08:38:17.884 INFO:teuthology.orchestra.run.smithi086.stdout:[osd.7] 2023-10-16T08:38:17.884 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQB19ixlyQj5OBAA6B/LtRfGS4w3CO0YNBWHnw== 2023-10-16T08:38:17.885 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2023-10-16T08:38:17.885 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBZ9ixluTq2NRAAu2d5FD9TEktWTvIi1OyVVg== 2023-10-16T08:38:17.885 INFO:teuthology.orchestra.run.smithi086.stdout:[client.2] 2023-10-16T08:38:17.885 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBZ9ixlHRolOxAAG/9KoJvTBAi4BNql0+I3xw== 2023-10-16T08:38:17.885 INFO:teuthology.orchestra.run.smithi086.stdout:[client.1] 2023-10-16T08:38:17.886 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBa9ixlcwFqAhAArt0fq+fo8Jny91S02+ewLA== 2023-10-16T08:38:17.886 INFO:teuthology.orchestra.run.smithi086.stdout:[client.3] 2023-10-16T08:38:17.886 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQBa9ixlgmKSBxAAw9uL06dASc+t66uiLzdaUw== 2023-10-16T08:38:17.887 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-16T08:38:17.923 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-16T08:38:17.972 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T08:38:17.975 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T08:38:18.063 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T08:38:18.066 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T08:38:18.112 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:38:18.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:38:18.191 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:38:18.199 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:38:18.274 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:38:18.283 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:38:18.356 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:38:18.371 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:38:18.436 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-16T08:38:18.459 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-16T08:38:18.525 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T08:38:18.548 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T08:38:18.615 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T08:38:18.631 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-16T08:38:18.679 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:38:18.681 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:38:18.759 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:38:18.772 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:38:18.839 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:38:18.842 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:38:18.920 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:38:18.923 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-16T08:38:19.000 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T08:38:19.011 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T08:38:19.083 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T08:38:19.086 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T08:38:19.170 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T08:38:19.173 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T08:38:19.217 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T08:38:19.259 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-16T08:38:19.301 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-16T08:38:19.301 DEBUG:teuthology.orchestra.run.smithi040:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-16T08:38:19.353 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-16T08:38:19.449 DEBUG:teuthology.orchestra.run.smithi040:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-16T08:38:19.460 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-16T08:38:19.473 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-16T08:38:19.571 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-16T08:38:19.582 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-16T08:38:19.640 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-16T08:38:19.739 DEBUG:teuthology.orchestra.run.smithi086:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-16T08:38:19.750 DEBUG:teuthology.orchestra.run.smithi040:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:38:19.753 DEBUG:teuthology.orchestra.run.smithi086:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-16T08:38:19.796 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-16T08:38:19.797 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-16T08:38:19.798 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-16T08:38:19.800 INFO:tasks.ceph.mon.a:Started 2023-10-16T08:38:19.800 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-16T08:38:19.801 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-16T08:38:19.842 INFO:tasks.ceph.mon.b:Started 2023-10-16T08:38:19.843 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-16T08:38:19.843 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-16T08:38:19.845 INFO:tasks.ceph.mon.c:Started 2023-10-16T08:38:19.845 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-16T08:38:19.846 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-16T08:38:19.846 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-16T08:38:19.849 INFO:tasks.ceph.mgr.y:Started 2023-10-16T08:38:19.849 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-16T08:38:19.849 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-16T08:38:19.851 INFO:tasks.ceph.mgr.x:Started 2023-10-16T08:38:19.852 INFO:tasks.ceph:Setting crush tunables to default 2023-10-16T08:38:19.852 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph osd crush tunables default 2023-10-16T08:38:19.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:19.969+0000 7f0596442640 1 Processor -- start 2023-10-16T08:38:19.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:19.969+0000 7f0596442640 1 -- start start 2023-10-16T08:38:19.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:19.973+0000 7f0596442640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f059015ad00 0x7f05900cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:19.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:19.973+0000 7f0596442640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05900cb800 0x7f0590065a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:19.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:19.973+0000 7f0596442640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05900b1b30 con 0x7f0590065fc0 2023-10-16T08:38:19.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:19.973+0000 7f0596442640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05900cbc80 con 0x7f059015ad00 2023-10-16T08:38:19.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:19.973+0000 7f0596442640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0590066450 con 0x7f05900cb800 2023-10-16T08:38:19.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:19.973+0000 7f05949df640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0590065fc0 0x7f059006bf00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60638/0 (socket says 172.21.15.40:60638) 2023-10-16T08:38:19.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:19.973+0000 7f05949df640 1 -- 172.21.15.40:0/375417793 learned_addr learned my addr 172.21.15.40:0/375417793 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:19.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:19.973+0000 7f058ffff640 1 --2- 172.21.15.40:0/375417793 >> v2:172.21.15.40:3300/0 conn(0x7f059015ad00 0x7f05900cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:19.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:19.973+0000 7f058ffff640 1 -- 172.21.15.40:0/375417793 >> v2:172.21.15.40:3300/0 conn(0x7f059015ad00 msgr2=0x7f05900cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-16T08:38:19.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:19.973+0000 7f058ffff640 1 -- 172.21.15.40:0/375417793 >> v2:172.21.15.40:3300/0 conn(0x7f059015ad00 msgr2=0x7f05900cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T08:38:19.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:19.973+0000 7f058ffff640 1 --2- 172.21.15.40:0/375417793 >> v2:172.21.15.40:3300/0 conn(0x7f059015ad00 0x7f05900cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T08:38:19.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:19.973+0000 7f058ffff640 1 --2- 172.21.15.40:0/375417793 >> v2:172.21.15.40:3300/0 conn(0x7f059015ad00 0x7f05900cb2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T08:38:19.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:19.973+0000 7f058f7fe640 1 --2- 172.21.15.40:0/375417793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05900cb800 0x7f0590065a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:19.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:19.973+0000 7f058f7fe640 1 -- 172.21.15.40:0/375417793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05900cb800 msgr2=0x7f0590065a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-16T08:38:19.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:19.973+0000 7f058f7fe640 1 -- 172.21.15.40:0/375417793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05900cb800 msgr2=0x7f0590065a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T08:38:19.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:19.973+0000 7f058f7fe640 1 --2- 172.21.15.40:0/375417793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05900cb800 0x7f0590065a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T08:38:19.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:19.973+0000 7f058f7fe640 1 --2- 172.21.15.40:0/375417793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05900cb800 0x7f0590065a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-16T08:38:20.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.177+0000 7f058ffff640 1 --2- 172.21.15.40:0/375417793 >> v2:172.21.15.40:3300/0 conn(0x7f059015ad00 0x7f05900cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:20.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.177+0000 7f058f7fe640 1 --2- 172.21.15.40:0/375417793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05900cb800 0x7f0590065a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:20.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.177+0000 7f058ffff640 1 -- 172.21.15.40:0/375417793 >> v2:172.21.15.40:3300/0 conn(0x7f059015ad00 msgr2=0x7f05900cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-16T08:38:20.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.177+0000 7f058ffff640 1 -- 172.21.15.40:0/375417793 >> v2:172.21.15.40:3300/0 conn(0x7f059015ad00 msgr2=0x7f05900cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T08:38:20.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.177+0000 7f058ffff640 1 --2- 172.21.15.40:0/375417793 >> v2:172.21.15.40:3300/0 conn(0x7f059015ad00 0x7f05900cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T08:38:20.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.177+0000 7f058ffff640 1 --2- 172.21.15.40:0/375417793 >> v2:172.21.15.40:3300/0 conn(0x7f059015ad00 0x7f05900cb2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T08:38:20.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.177+0000 7f058f7fe640 1 -- 172.21.15.40:0/375417793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05900cb800 msgr2=0x7f0590065a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-16T08:38:20.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.177+0000 7f058f7fe640 1 -- 172.21.15.40:0/375417793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05900cb800 msgr2=0x7f0590065a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-16T08:38:20.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.177+0000 7f058f7fe640 1 --2- 172.21.15.40:0/375417793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05900cb800 0x7f0590065a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-10-16T08:38:20.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.177+0000 7f058f7fe640 1 --2- 172.21.15.40:0/375417793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05900cb800 0x7f0590065a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-10-16T08:38:20.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.233+0000 7f058effd640 1 -- 172.21.15.40:0/375417793 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2925271121 0 0) 0x7f05900b1b30 con 0x7f0590065fc0 2023-10-16T08:38:20.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.233+0000 7f058effd640 1 -- 172.21.15.40:0/375417793 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0578003530 con 0x7f0590065fc0 2023-10-16T08:38:20.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.233+0000 7f058effd640 1 -- 172.21.15.40:0/375417793 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2485657054 0 0) 0x7f0578003530 con 0x7f0590065fc0 2023-10-16T08:38:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.233+0000 7f058effd640 1 -- 172.21.15.40:0/375417793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05900cb800 msgr2=0x7f0590065a80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T08:38:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.233+0000 7f058effd640 1 --2- 172.21.15.40:0/375417793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05900cb800 0x7f0590065a80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.233+0000 7f058effd640 1 -- 172.21.15.40:0/375417793 >> v2:172.21.15.40:3300/0 conn(0x7f059015ad00 msgr2=0x7f05900cb2c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T08:38:20.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.233+0000 7f058effd640 1 --2- 172.21.15.40:0/375417793 >> v2:172.21.15.40:3300/0 conn(0x7f059015ad00 0x7f05900cb2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.233+0000 7f058effd640 1 -- 172.21.15.40:0/375417793 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05900cbe80 con 0x7f0590065fc0 2023-10-16T08:38:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.233+0000 7f058effd640 1 -- 172.21.15.40:0/375417793 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f0588001a00 con 0x7f0590065fc0 2023-10-16T08:38:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.233+0000 7f058effd640 1 -- 172.21.15.40:0/375417793 >> v1:172.21.15.40:6789/0 conn(0x7f0590065fc0 legacy=0x7f059006bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.233+0000 7f058effd640 1 --2- 172.21.15.40:0/375417793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05780039f0 0x7f0578003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.233+0000 7f058effd640 1 --2- 172.21.15.40:0/375417793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05900cb800 0x7f05780047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.233+0000 7f058effd640 1 --2- 172.21.15.40:0/375417793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0578004d10 0x7f0578005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:20.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.233+0000 7f058effd640 1 -- 172.21.15.40:0/375417793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05900cbc80 con 0x7f05780039f0 2023-10-16T08:38:20.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.233+0000 7f058effd640 1 -- 172.21.15.40:0/375417793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0590066450 con 0x7f0578004d10 2023-10-16T08:38:20.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.233+0000 7f058effd640 1 -- 172.21.15.40:0/375417793 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f057800ba10 con 0x7f05900cb800 2023-10-16T08:38:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.237+0000 7f058ffff640 1 --2- 172.21.15.40:0/375417793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05780039f0 0x7f0578003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.237+0000 7f05949df640 1 --2- 172.21.15.40:0/375417793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0578004d10 0x7f0578005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.237+0000 7f058f7fe640 1 --2- 172.21.15.40:0/375417793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05900cb800 0x7f05780047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.237+0000 7f058ffff640 1 -- 172.21.15.40:0/375417793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05900cb800 msgr2=0x7f05780047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:20.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.237+0000 7f058ffff640 1 --2- 172.21.15.40:0/375417793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f05900cb800 0x7f05780047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:20.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.237+0000 7f058ffff640 1 -- 172.21.15.40:0/375417793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0578004d10 msgr2=0x7f0578005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:20.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.237+0000 7f058ffff640 1 --2- 172.21.15.40:0/375417793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0578004d10 0x7f0578005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:20.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.237+0000 7f058ffff640 1 -- 172.21.15.40:0/375417793 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0578003530 con 0x7f05780039f0 2023-10-16T08:38:20.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.241+0000 7f058ffff640 1 --2- 172.21.15.40:0/375417793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05780039f0 0x7f0578003dd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f058000e8d0 tx=0x7f058001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4e1c1952f53ea63 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:20.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.249+0000 7f058effd640 1 -- 172.21.15.40:0/375417793 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05800a9660 con 0x7f05780039f0 2023-10-16T08:38:20.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.249+0000 7f058effd640 1 -- 172.21.15.40:0/375417793 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05800a97c0 con 0x7f05780039f0 2023-10-16T08:38:20.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.249+0000 7f058effd640 1 -- 172.21.15.40:0/375417793 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05800a9b70 con 0x7f05780039f0 2023-10-16T08:38:20.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.249+0000 7f0596442640 1 -- 172.21.15.40:0/375417793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05780039f0 msgr2=0x7f0578003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:20.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.249+0000 7f0596442640 1 --2- 172.21.15.40:0/375417793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05780039f0 0x7f0578003dd0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f058000e8d0 tx=0x7f058001c890 comp rx=0 tx=0).stop 2023-10-16T08:38:20.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.249+0000 7f05949df640 1 -- 172.21.15.40:0/375417793 reap_dead start 2023-10-16T08:38:20.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.249+0000 7f0596442640 1 -- 172.21.15.40:0/375417793 shutdown_connections 2023-10-16T08:38:20.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.249+0000 7f0596442640 1 -- 172.21.15.40:0/375417793 >> 172.21.15.40:0/375417793 conn(0x7f05900cc2d0 msgr2=0x7f05900cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:20.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.249+0000 7f0596442640 1 -- 172.21.15.40:0/375417793 shutdown_connections 2023-10-16T08:38:20.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.249+0000 7f0596442640 1 -- 172.21.15.40:0/375417793 wait complete. 2023-10-16T08:38:20.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.249+0000 7f0596442640 1 Processor -- start 2023-10-16T08:38:20.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.249+0000 7f0596442640 1 -- start start 2023-10-16T08:38:20.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.249+0000 7f0596442640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05780039f0 0x7f059014b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:20.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.249+0000 7f0596442640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0590157bd0 0x7f059015b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:20.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.249+0000 7f0596442640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05900b2f70 0x7f0590147ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:20.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.249+0000 7f0596442640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059015b8f0 con 0x7f05900b2f70 2023-10-16T08:38:20.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.249+0000 7f0596442640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059015bb20 con 0x7f05780039f0 2023-10-16T08:38:20.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.249+0000 7f0596442640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059014ac40 con 0x7f0590157bd0 2023-10-16T08:38:20.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.253+0000 7f05949df640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05900b2f70 0x7f0590147ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:20.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.253+0000 7f05949df640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05900b2f70 0x7f0590147ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50636/0 (socket says 172.21.15.40:50636) 2023-10-16T08:38:20.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.253+0000 7f05949df640 1 -- 172.21.15.40:0/1220571416 learned_addr learned my addr 172.21.15.40:0/1220571416 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:20.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.253+0000 7f058ffff640 1 --2- 172.21.15.40:0/1220571416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05780039f0 0x7f059014b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:20.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.253+0000 7f058f7fe640 1 --2- 172.21.15.40:0/1220571416 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0590157bd0 0x7f059015b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:20.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.253+0000 7f058f7fe640 1 -- 172.21.15.40:0/1220571416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05780039f0 msgr2=0x7f059014b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:20.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.253+0000 7f058f7fe640 1 --2- 172.21.15.40:0/1220571416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05780039f0 0x7f059014b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:20.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.253+0000 7f058f7fe640 1 -- 172.21.15.40:0/1220571416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05900b2f70 msgr2=0x7f0590147ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:20.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.253+0000 7f058f7fe640 1 --2- 172.21.15.40:0/1220571416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05900b2f70 0x7f0590147ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:20.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.253+0000 7f058f7fe640 1 -- 172.21.15.40:0/1220571416 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f059006ded0 con 0x7f0590157bd0 2023-10-16T08:38:20.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.253+0000 7f05949df640 1 --2- 172.21.15.40:0/1220571416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05900b2f70 0x7f0590147ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:38:20.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.253+0000 7f058f7fe640 1 --2- 172.21.15.40:0/1220571416 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0590157bd0 0x7f059015b3b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0584013700 tx=0x7f0584013b00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:20.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.253+0000 7f058cff9640 1 -- 172.21.15.40:0/1220571416 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584004500 con 0x7f0590157bd0 2023-10-16T08:38:20.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.253+0000 7f058cff9640 1 -- 172.21.15.40:0/1220571416 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f058400c070 con 0x7f0590157bd0 2023-10-16T08:38:20.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.253+0000 7f0596442640 1 -- 172.21.15.40:0/1220571416 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f059014af20 con 0x7f0590157bd0 2023-10-16T08:38:20.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.253+0000 7f0596442640 1 -- 172.21.15.40:0/1220571416 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f059015be00 con 0x7f0590157bd0 2023-10-16T08:38:20.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.253+0000 7f058cff9640 1 -- 172.21.15.40:0/1220571416 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0584003840 con 0x7f0590157bd0 2023-10-16T08:38:20.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.253+0000 7f058cff9640 1 -- 172.21.15.40:0/1220571416 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0584003d20 con 0x7f0590157bd0 2023-10-16T08:38:20.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.253+0000 7f058cff9640 1 -- 172.21.15.40:0/1220571416 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f0584006550 con 0x7f0590157bd0 2023-10-16T08:38:20.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.253+0000 7f0596442640 1 -- 172.21.15.40:0/1220571416 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f059004e9c0 con 0x7f0590157bd0 2023-10-16T08:38:20.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.317+0000 7f058cff9640 1 -- 172.21.15.40:0/1220571416 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f058400e050 con 0x7f0590157bd0 2023-10-16T08:38:20.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.385+0000 7f0596442640 1 -- 172.21.15.40:0/1220571416 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f05900af520 con 0x7f0590157bd0 2023-10-16T08:38:20.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.393+0000 7f058cff9640 1 -- 172.21.15.40:0/1220571416 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f058400f040 con 0x7f0590157bd0 2023-10-16T08:38:20.397 INFO:teuthology.orchestra.run.smithi040.stderr:adjusted tunables profile to default 2023-10-16T08:38:20.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.397+0000 7f0596442640 1 -- 172.21.15.40:0/1220571416 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0590157bd0 msgr2=0x7f059015b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:20.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.397+0000 7f0596442640 1 --2- 172.21.15.40:0/1220571416 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0590157bd0 0x7f059015b3b0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0584013700 tx=0x7f0584013b00 comp rx=0 tx=0).stop 2023-10-16T08:38:20.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.397+0000 7f0596442640 1 -- 172.21.15.40:0/1220571416 shutdown_connections 2023-10-16T08:38:20.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.397+0000 7f0596442640 1 --2- 172.21.15.40:0/1220571416 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f05900b2f70 0x7f0590147ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:20.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.397+0000 7f0596442640 1 --2- 172.21.15.40:0/1220571416 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0590157bd0 0x7f059015b3b0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:20.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.397+0000 7f0596442640 1 --2- 172.21.15.40:0/1220571416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05780039f0 0x7f059014b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:20.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.397+0000 7f0596442640 1 -- 172.21.15.40:0/1220571416 >> 172.21.15.40:0/1220571416 conn(0x7f05900cc2d0 msgr2=0x7f059006bf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:20.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.397+0000 7f0596442640 1 -- 172.21.15.40:0/1220571416 shutdown_connections 2023-10-16T08:38:20.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:20.397+0000 7f0596442640 1 -- 172.21.15.40:0/1220571416 wait complete. 2023-10-16T08:38:20.419 INFO:tasks.ceph:check_enable_crimson: False 2023-10-16T08:38:20.419 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-16T08:38:20.419 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:38:20.420 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-16T08:38:20.432 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:38:20.433 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-16T08:38:20.490 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:38:20.491 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-16T08:38:20.539 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:20.537+0000 7f5901f19180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T08:38:20.546 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:20.546+0000 7f284cdce180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-16T08:38:20.549 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:38:20.550 DEBUG:teuthology.orchestra.run.smithi040:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-16T08:38:20.610 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:38:20.610 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-16T08:38:20.624 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:38:20.624 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-16T08:38:20.676 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:38:20.677 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-16T08:38:20.732 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:38:20.733 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-16T08:38:20.788 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new b731cf12-d90b-4411-b989-7bd8fa503126 0 2023-10-16T08:38:20.951 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.950+0000 7fe41a1a4640 1 Processor -- start 2023-10-16T08:38:20.952 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.950+0000 7fe41a1a4640 1 -- start start 2023-10-16T08:38:20.953 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.950+0000 7fe41a1a4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe4140ce250 0x7fe4140b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:20.953 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.950+0000 7fe41a1a4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4140b1c80 con 0x7fe4140cde70 2023-10-16T08:38:20.953 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.950+0000 7fe41a1a4640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe41406c580 con 0x7fe4140b1810 2023-10-16T08:38:20.954 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.950+0000 7fe41a1a4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe41406c860 con 0x7fe4140ce250 2023-10-16T08:38:20.954 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.950+0000 7fe412ffd640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe4140ce250 0x7fe4140b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:20.954 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.950+0000 7fe413fff640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fe4140b1810 0x7fe4141594a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44402/0 (socket says 172.21.15.86:44402) 2023-10-16T08:38:20.954 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.950+0000 7fe413fff640 1 -- 172.21.15.86:0/2519738840 learned_addr learned my addr 172.21.15.86:0/2519738840 (peer_addr_for_me v1:172.21.15.86:0/0) 2023-10-16T08:38:20.954 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.950+0000 7fe412ffd640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe4140ce250 0x7fe4140b12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:48634/0 (socket says 172.21.15.86:48634) 2023-10-16T08:38:20.955 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.954+0000 7fe4127fc640 1 -- 172.21.15.86:0/2519738840 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 676907921 0 0) 0x7fe41406c580 con 0x7fe4140b1810 2023-10-16T08:38:20.955 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.954+0000 7fe412ffd640 1 -- 172.21.15.86:0/2519738840 >> v1:172.21.15.86:6789/0 conn(0x7fe4140b1810 legacy=0x7fe4141594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:20.955 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.954+0000 7fe412ffd640 1 -- 172.21.15.86:0/2519738840 >> v1:172.21.15.40:6789/0 conn(0x7fe4140cde70 legacy=0x7fe41406d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:20.955 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.954+0000 7fe412ffd640 1 -- 172.21.15.86:0/2519738840 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe41406cb40 con 0x7fe4140ce250 2023-10-16T08:38:20.956 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.954+0000 7fe412ffd640 1 --2- 172.21.15.86:0/2519738840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe4140ce250 0x7fe4140b12d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00ba90 tx=0x7fe3fc01eae0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4a861676287eb83 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:20.957 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.954+0000 7fe4127fc640 1 -- 172.21.15.86:0/2519738840 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc0a7bd0 con 0x7fe4140ce250 2023-10-16T08:38:20.957 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.954+0000 7fe4127fc640 1 -- 172.21.15.86:0/2519738840 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3fc0a7d30 con 0x7fe4140ce250 2023-10-16T08:38:20.957 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.954+0000 7fe4127fc640 1 -- 172.21.15.86:0/2519738840 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc0a7ea0 con 0x7fe4140ce250 2023-10-16T08:38:20.957 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.954+0000 7fe41a1a4640 1 -- 172.21.15.86:0/2519738840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe4140ce250 msgr2=0x7fe4140b12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:20.957 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.954+0000 7fe41a1a4640 1 --2- 172.21.15.86:0/2519738840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe4140ce250 0x7fe4140b12d0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc00ba90 tx=0x7fe3fc01eae0 comp rx=0 tx=0).stop 2023-10-16T08:38:20.958 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.954+0000 7fe41a1a4640 1 -- 172.21.15.86:0/2519738840 shutdown_connections 2023-10-16T08:38:20.958 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.954+0000 7fe41a1a4640 1 --2- 172.21.15.86:0/2519738840 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe4140ce250 0x7fe4140b12d0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:20.958 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.954+0000 7fe41a1a4640 1 -- 172.21.15.86:0/2519738840 >> 172.21.15.86:0/2519738840 conn(0x7fe4140cc2d0 msgr2=0x7fe4140cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:20.958 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.954+0000 7fe41a1a4640 1 -- 172.21.15.86:0/2519738840 shutdown_connections 2023-10-16T08:38:20.958 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.954+0000 7fe41a1a4640 1 -- 172.21.15.86:0/2519738840 wait complete. 2023-10-16T08:38:20.961 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe41a1a4640 1 Processor -- start 2023-10-16T08:38:20.961 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe41a1a4640 1 -- start start 2023-10-16T08:38:20.961 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe41a1a4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe4140b1810 0x7fe414153bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:20.961 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe41a1a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4140cde70 0x7fe414154110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:20.961 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe41a1a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4140ce250 0x7fe4140b2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:20.962 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe41a1a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe414151ac0 con 0x7fe4140cde70 2023-10-16T08:38:20.962 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe41a1a4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe414151cc0 con 0x7fe4140ce250 2023-10-16T08:38:20.962 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe41a1a4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe414151fa0 con 0x7fe4140b1810 2023-10-16T08:38:20.962 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe4137fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe4140b1810 0x7fe414153bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:20.962 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe413fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4140ce250 0x7fe4140b2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:20.963 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe4137fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe4140b1810 0x7fe414153bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:48650/0 (socket says 172.21.15.86:48650) 2023-10-16T08:38:20.963 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe4137fe640 1 -- 172.21.15.86:0/1624146238 learned_addr learned my addr 172.21.15.86:0/1624146238 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-16T08:38:20.963 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe412ffd640 1 --2- 172.21.15.86:0/1624146238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4140cde70 0x7fe414154110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:20.963 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe4137fe640 1 -- 172.21.15.86:0/1624146238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4140ce250 msgr2=0x7fe4140b2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:20.963 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe4137fe640 1 --2- 172.21.15.86:0/1624146238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4140ce250 0x7fe4140b2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:20.964 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe4137fe640 1 -- 172.21.15.86:0/1624146238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4140cde70 msgr2=0x7fe414154110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:20.964 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe4137fe640 1 --2- 172.21.15.86:0/1624146238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4140cde70 0x7fe414154110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:20.964 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe4137fe640 1 -- 172.21.15.86:0/1624146238 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3fc00b6c0 con 0x7fe4140b1810 2023-10-16T08:38:20.964 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe413fff640 1 --2- 172.21.15.86:0/1624146238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4140ce250 0x7fe4140b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:38:20.964 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe4137fe640 1 --2- 172.21.15.86:0/1624146238 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe4140b1810 0x7fe414153bd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00c500 tx=0x7fe40c00c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:20.965 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe40bfff640 1 -- 172.21.15.86:0/1624146238 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c005220 con 0x7fe4140b1810 2023-10-16T08:38:20.965 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe40bfff640 1 -- 172.21.15.86:0/1624146238 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe40c005380 con 0x7fe4140b1810 2023-10-16T08:38:20.965 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe41a1a4640 1 -- 172.21.15.86:0/1624146238 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe414152280 con 0x7fe4140b1810 2023-10-16T08:38:20.965 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe40bfff640 1 -- 172.21.15.86:0/1624146238 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe40c007270 con 0x7fe4140b1810 2023-10-16T08:38:20.965 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe41a1a4640 1 -- 172.21.15.86:0/1624146238 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe414154f80 con 0x7fe4140b1810 2023-10-16T08:38:20.965 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe40bfff640 1 -- 172.21.15.86:0/1624146238 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fe40c0067e0 con 0x7fe4140b1810 2023-10-16T08:38:20.966 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe40bfff640 1 -- 172.21.15.86:0/1624146238 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe40c0129a0 con 0x7fe4140b1810 2023-10-16T08:38:20.966 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.958+0000 7fe41a1a4640 1 -- 172.21.15.86:0/1624146238 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe41404e9c0 con 0x7fe4140b1810 2023-10-16T08:38:20.966 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:20.962+0000 7fe40bfff640 1 -- 172.21.15.86:0/1624146238 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fe40c016020 con 0x7fe4140b1810 2023-10-16T08:38:21.031 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.030+0000 7fe41a1a4640 1 -- 172.21.15.86:0/1624146238 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b731cf12-d90b-4411-b989-7bd8fa503126", "id": 0} v 0) v1 -- 0x7fe4140ccb70 con 0x7fe4140b1810 2023-10-16T08:38:21.039 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.038+0000 7fe40bfff640 1 -- 172.21.15.86:0/1624146238 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b731cf12-d90b-4411-b989-7bd8fa503126", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7fe40c02a0b0 con 0x7fe4140b1810 2023-10-16T08:38:21.040 INFO:teuthology.orchestra.run.smithi086.stdout:0 2023-10-16T08:38:21.041 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.038+0000 7fe41a1a4640 1 -- 172.21.15.86:0/1624146238 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe4140b1810 msgr2=0x7fe414153bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.041 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.038+0000 7fe41a1a4640 1 --2- 172.21.15.86:0/1624146238 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe4140b1810 0x7fe414153bd0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe40c00c500 tx=0x7fe40c00c9c0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.041 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.038+0000 7fe41a1a4640 1 -- 172.21.15.86:0/1624146238 shutdown_connections 2023-10-16T08:38:21.042 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.038+0000 7fe41a1a4640 1 --2- 172.21.15.86:0/1624146238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe4140ce250 0x7fe4140b2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.042 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.038+0000 7fe41a1a4640 1 --2- 172.21.15.86:0/1624146238 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe4140cde70 0x7fe414154110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.042 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.038+0000 7fe41a1a4640 1 --2- 172.21.15.86:0/1624146238 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe4140b1810 0x7fe414153bd0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.042 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.038+0000 7fe41a1a4640 1 -- 172.21.15.86:0/1624146238 >> 172.21.15.86:0/1624146238 conn(0x7fe4140cc2d0 msgr2=0x7fe41406d340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:21.042 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.038+0000 7fe41a1a4640 1 -- 172.21.15.86:0/1624146238 shutdown_connections 2023-10-16T08:38:21.043 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.042+0000 7fe41a1a4640 1 -- 172.21.15.86:0/1624146238 wait complete. 2023-10-16T08:38:21.057 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 994268ab-dc28-42c8-8f2a-38eb2aedd884 1 2023-10-16T08:38:21.081 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:21.082+0000 7f284cdce180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T08:38:21.115 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:21.113+0000 7f5901f19180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-16T08:38:21.178 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c8ed6d640 1 Processor -- start 2023-10-16T08:38:21.179 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c8ed6d640 1 -- start start 2023-10-16T08:38:21.181 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c8ed6d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c8815cf00 0x7f3c881609e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.181 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c8ed6d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c880d0c50 con 0x7f3c88160f20 2023-10-16T08:38:21.181 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c8ed6d640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c8815d2e0 con 0x7f3c880b5fa0 2023-10-16T08:38:21.182 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c8ed6d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c88161480 con 0x7f3c8815cf00 2023-10-16T08:38:21.182 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c8d30a640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f3c880b5fa0 0x7f3c880cd9d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44406/0 (socket says 172.21.15.86:44406) 2023-10-16T08:38:21.182 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c8d30a640 1 -- 172.21.15.86:0/185474555 learned_addr learned my addr 172.21.15.86:0/185474555 (peer_addr_for_me v1:172.21.15.86:0/0) 2023-10-16T08:38:21.182 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c8cb09640 1 --2- 172.21.15.86:0/185474555 >> v2:172.21.15.40:3300/0 conn(0x7f3c8815cf00 0x7f3c881609e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.182 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c877fe640 1 -- 172.21.15.86:0/185474555 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1763550799 0 0) 0x7f3c8815d2e0 con 0x7f3c880b5fa0 2023-10-16T08:38:21.183 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c877fe640 1 -- 172.21.15.86:0/185474555 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c74003530 con 0x7f3c880b5fa0 2023-10-16T08:38:21.183 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c877fe640 1 -- 172.21.15.86:0/185474555 <== mon.1 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 897162777 0 0) 0x7f3c74003530 con 0x7f3c880b5fa0 2023-10-16T08:38:21.183 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c877fe640 1 -- 172.21.15.86:0/185474555 >> v2:172.21.15.40:3300/0 conn(0x7f3c8815cf00 msgr2=0x7f3c881609e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:21.183 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c877fe640 1 --2- 172.21.15.86:0/185474555 >> v2:172.21.15.40:3300/0 conn(0x7f3c8815cf00 0x7f3c881609e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.183 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c877fe640 1 -- 172.21.15.86:0/185474555 >> v1:172.21.15.40:6789/0 conn(0x7f3c88160f20 legacy=0x7f3c880d16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.183 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c877fe640 1 -- 172.21.15.86:0/185474555 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c88161760 con 0x7f3c880b5fa0 2023-10-16T08:38:21.184 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c877fe640 1 -- 172.21.15.86:0/185474555 <== mon.1 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f3c80001a00 con 0x7f3c880b5fa0 2023-10-16T08:38:21.184 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c877fe640 1 -- 172.21.15.86:0/185474555 >> v1:172.21.15.86:6789/0 conn(0x7f3c880b5fa0 legacy=0x7f3c880cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.184 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c877fe640 1 --2- 172.21.15.86:0/185474555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c740039f0 0x7f3c74003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.184 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c877fe640 1 --2- 172.21.15.86:0/185474555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c740047d0 0x7f3c74004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.184 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c877fe640 1 --2- 172.21.15.86:0/185474555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c74005200 0x7f3c7400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.185 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c877fe640 1 -- 172.21.15.86:0/185474555 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c88161480 con 0x7f3c740047d0 2023-10-16T08:38:21.185 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c877fe640 1 -- 172.21.15.86:0/185474555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7400c160 con 0x7f3c74005200 2023-10-16T08:38:21.185 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c877fe640 1 -- 172.21.15.86:0/185474555 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c7400c320 con 0x7f3c740039f0 2023-10-16T08:38:21.185 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c8cb09640 1 --2- 172.21.15.86:0/185474555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c740039f0 0x7f3c74003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.185 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c8d30a640 1 --2- 172.21.15.86:0/185474555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c74005200 0x7f3c7400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.186 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c87fff640 1 --2- 172.21.15.86:0/185474555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c740047d0 0x7f3c74004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.186 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c8d30a640 1 -- 172.21.15.86:0/185474555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c740039f0 msgr2=0x7f3c74003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:21.186 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c8d30a640 1 --2- 172.21.15.86:0/185474555 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c740039f0 0x7f3c74003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.186 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c8d30a640 1 -- 172.21.15.86:0/185474555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c740047d0 msgr2=0x7f3c74004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:21.186 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c8d30a640 1 --2- 172.21.15.86:0/185474555 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c740047d0 0x7f3c74004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.187 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.177+0000 7f3c8d30a640 1 -- 172.21.15.86:0/185474555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c74003530 con 0x7f3c74005200 2023-10-16T08:38:21.190 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.181+0000 7f3c8d30a640 1 -- 172.21.15.86:0/185474555 reap_dead start 2023-10-16T08:38:21.190 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.181+0000 7f3c8d30a640 1 --2- 172.21.15.86:0/185474555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c74005200 0x7f3c7400b990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000a830 tx=0x7f3c8001d250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f50ce77a70326668 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:21.190 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.181+0000 7f3c877fe640 1 -- 172.21.15.86:0/185474555 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c800072d0 con 0x7f3c74005200 2023-10-16T08:38:21.191 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.181+0000 7f3c877fe640 1 -- 172.21.15.86:0/185474555 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c800a8070 con 0x7f3c74005200 2023-10-16T08:38:21.191 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.181+0000 7f3c877fe640 1 -- 172.21.15.86:0/185474555 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c800076b0 con 0x7f3c74005200 2023-10-16T08:38:21.191 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.181+0000 7f3c8ed6d640 1 -- 172.21.15.86:0/185474555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c74005200 msgr2=0x7f3c7400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.191 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.181+0000 7f3c8ed6d640 1 --2- 172.21.15.86:0/185474555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c74005200 0x7f3c7400b990 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3c8000a830 tx=0x7f3c8001d250 comp rx=0 tx=0).stop 2023-10-16T08:38:21.191 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.181+0000 7f3c8ed6d640 1 -- 172.21.15.86:0/185474555 shutdown_connections 2023-10-16T08:38:21.192 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.181+0000 7f3c8ed6d640 1 --2- 172.21.15.86:0/185474555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c74005200 0x7f3c7400b990 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.192 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.181+0000 7f3c8ed6d640 1 -- 172.21.15.86:0/185474555 >> 172.21.15.86:0/185474555 conn(0x7f3c88065a80 msgr2=0x7f3c88065e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:21.192 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.181+0000 7f3c8ed6d640 1 -- 172.21.15.86:0/185474555 shutdown_connections 2023-10-16T08:38:21.192 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.181+0000 7f3c8ed6d640 1 -- 172.21.15.86:0/185474555 wait complete. 2023-10-16T08:38:21.192 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8ed6d640 1 Processor -- start 2023-10-16T08:38:21.192 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8ed6d640 1 -- start start 2023-10-16T08:38:21.193 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8ed6d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c74005200 0x7f3c8815f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.193 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8ed6d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c880c5580 0x7f3c881f5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.193 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8ed6d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c881f59b0 0x7f3c88152940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.193 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8ed6d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c881524b0 con 0x7f3c881f59b0 2023-10-16T08:38:21.193 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8ed6d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8814c1f0 con 0x7f3c880c5580 2023-10-16T08:38:21.194 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8ed6d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c8814deb0 con 0x7f3c74005200 2023-10-16T08:38:21.194 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8cb09640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c74005200 0x7f3c8815f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.194 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8cb09640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c74005200 0x7f3c8815f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:48670/0 (socket says 172.21.15.86:48670) 2023-10-16T08:38:21.194 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8cb09640 1 -- 172.21.15.86:0/191786433 learned_addr learned my addr 172.21.15.86:0/191786433 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-16T08:38:21.194 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c87fff640 1 --2- 172.21.15.86:0/191786433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c880c5580 0x7f3c881f5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.195 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8d30a640 1 --2- 172.21.15.86:0/191786433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c881f59b0 0x7f3c88152940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.195 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8cb09640 1 -- 172.21.15.86:0/191786433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c880c5580 msgr2=0x7f3c881f5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.195 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8cb09640 1 --2- 172.21.15.86:0/191786433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c880c5580 0x7f3c881f5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.195 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8cb09640 1 -- 172.21.15.86:0/191786433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c881f59b0 msgr2=0x7f3c88152940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.195 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8cb09640 1 --2- 172.21.15.86:0/191786433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c881f59b0 0x7f3c88152940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.195 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8cb09640 1 -- 172.21.15.86:0/191786433 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c8814d630 con 0x7f3c74005200 2023-10-16T08:38:21.196 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c87fff640 1 --2- 172.21.15.86:0/191786433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c880c5580 0x7f3c881f5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:38:21.196 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8d30a640 1 --2- 172.21.15.86:0/191786433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c881f59b0 0x7f3c88152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:38:21.196 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8cb09640 1 --2- 172.21.15.86:0/191786433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c74005200 0x7f3c8815f780 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3c78011960 tx=0x7f3c78011e20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:21.196 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c857fa640 1 -- 172.21.15.86:0/191786433 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c7800faf0 con 0x7f3c74005200 2023-10-16T08:38:21.196 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c857fa640 1 -- 172.21.15.86:0/191786433 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c7800fc50 con 0x7f3c74005200 2023-10-16T08:38:21.196 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8ed6d640 1 -- 172.21.15.86:0/191786433 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c8814d8f0 con 0x7f3c74005200 2023-10-16T08:38:21.197 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c857fa640 1 -- 172.21.15.86:0/191786433 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c78014440 con 0x7f3c74005200 2023-10-16T08:38:21.197 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8ed6d640 1 -- 172.21.15.86:0/191786433 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c880c5a00 con 0x7f3c74005200 2023-10-16T08:38:21.197 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c857fa640 1 -- 172.21.15.86:0/191786433 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f3c780148a0 con 0x7f3c74005200 2023-10-16T08:38:21.197 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c857fa640 1 -- 172.21.15.86:0/191786433 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f3c7801dc90 con 0x7f3c74005200 2023-10-16T08:38:21.197 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.185+0000 7f3c8ed6d640 1 -- 172.21.15.86:0/191786433 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c8804e9c0 con 0x7f3c74005200 2023-10-16T08:38:21.198 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.189+0000 7f3c857fa640 1 -- 172.21.15.86:0/191786433 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f3c7801d400 con 0x7f3c74005200 2023-10-16T08:38:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.261+0000 7f3c8ed6d640 1 -- 172.21.15.86:0/191786433 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd new", "uuid": "994268ab-dc28-42c8-8f2a-38eb2aedd884", "id": 1} v 0) v1 -- 0x7f3c880d0a30 con 0x7f3c74005200 2023-10-16T08:38:21.268 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.265+0000 7f3c857fa640 1 -- 172.21.15.86:0/191786433 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "994268ab-dc28-42c8-8f2a-38eb2aedd884", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3c78014c90 con 0x7f3c74005200 2023-10-16T08:38:21.268 INFO:teuthology.orchestra.run.smithi086.stdout:1 2023-10-16T08:38:21.269 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.269+0000 7f3c8ed6d640 1 -- 172.21.15.86:0/191786433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c74005200 msgr2=0x7f3c8815f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.270 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.269+0000 7f3c8ed6d640 1 --2- 172.21.15.86:0/191786433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c74005200 0x7f3c8815f780 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3c78011960 tx=0x7f3c78011e20 comp rx=0 tx=0).stop 2023-10-16T08:38:21.270 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.269+0000 7f3c8ed6d640 1 -- 172.21.15.86:0/191786433 shutdown_connections 2023-10-16T08:38:21.270 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.269+0000 7f3c8ed6d640 1 --2- 172.21.15.86:0/191786433 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c881f59b0 0x7f3c88152940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.270 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.269+0000 7f3c8ed6d640 1 --2- 172.21.15.86:0/191786433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c880c5580 0x7f3c881f5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.270 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.269+0000 7f3c8ed6d640 1 --2- 172.21.15.86:0/191786433 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c74005200 0x7f3c8815f780 secure :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3c78011960 tx=0x7f3c78011e20 comp rx=0 tx=0).stop 2023-10-16T08:38:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.269+0000 7f3c8ed6d640 1 -- 172.21.15.86:0/191786433 >> 172.21.15.86:0/191786433 conn(0x7f3c88065a80 msgr2=0x7f3c880b7a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.269+0000 7f3c8ed6d640 1 -- 172.21.15.86:0/191786433 shutdown_connections 2023-10-16T08:38:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.269+0000 7f3c8ed6d640 1 -- 172.21.15.86:0/191786433 wait complete. 2023-10-16T08:38:21.281 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:21.281+0000 7f284cdce180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T08:38:21.285 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32 2 2023-10-16T08:38:21.447 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.445+0000 7fc092af2640 1 Processor -- start 2023-10-16T08:38:21.447 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.445+0000 7fc092af2640 1 -- start start 2023-10-16T08:38:21.449 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.445+0000 7fc092af2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc08c0b4500 0x7fc08c0cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.449 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.445+0000 7fc092af2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc08c0cdc10 0x7fc08c06d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.449 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.445+0000 7fc092af2640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc08c06fba0 con 0x7fc08c06df00 2023-10-16T08:38:21.449 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.445+0000 7fc092af2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc08c15b120 con 0x7fc08c0cdc10 2023-10-16T08:38:21.449 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.445+0000 7fc092af2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c06c1b0 con 0x7fc08c0b4500 2023-10-16T08:38:21.450 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.445+0000 7fc09088e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc08c0b4500 0x7fc08c0cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.450 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.445+0000 7fc09088e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc08c0b4500 0x7fc08c0cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:48686/0 (socket says 172.21.15.86:48686) 2023-10-16T08:38:21.450 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.445+0000 7fc09088e640 1 -- 172.21.15.86:0/3024980225 learned_addr learned my addr 172.21.15.86:0/3024980225 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-16T08:38:21.450 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.445+0000 7fc08bfff640 1 --2- 172.21.15.86:0/3024980225 >> v2:172.21.15.40:3300/0 conn(0x7fc08c0cdc10 0x7fc08c06d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.450 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.445+0000 7fc09088e640 1 -- 172.21.15.86:0/3024980225 >> v2:172.21.15.40:3300/0 conn(0x7fc08c0cdc10 msgr2=0x7fc08c06d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:21.451 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.445+0000 7fc09088e640 1 --2- 172.21.15.86:0/3024980225 >> v2:172.21.15.40:3300/0 conn(0x7fc08c0cdc10 0x7fc08c06d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.451 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.445+0000 7fc09088e640 1 -- 172.21.15.86:0/3024980225 >> v1:172.21.15.40:6789/0 conn(0x7fc08c06df00 legacy=0x7fc08c15e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:21.451 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.445+0000 7fc09088e640 1 -- 172.21.15.86:0/3024980225 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08c070160 con 0x7fc08c0b4500 2023-10-16T08:38:21.451 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.445+0000 7fc08bfff640 1 --2- 172.21.15.86:0/3024980225 >> v2:172.21.15.40:3300/0 conn(0x7fc08c0cdc10 0x7fc08c06d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:21.452 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:21.445+0000 7f5901f19180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-16T08:38:21.453 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.449+0000 7fc09088e640 1 --2- 172.21.15.86:0/3024980225 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc08c0b4500 0x7fc08c0cd6d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc084009e50 tx=0x7fc08401c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=707b51fd9e9c2766 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:21.453 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.449+0000 7fc08b7fe640 1 -- 172.21.15.86:0/3024980225 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0840a5860 con 0x7fc08c0b4500 2023-10-16T08:38:21.454 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.449+0000 7fc08b7fe640 1 -- 172.21.15.86:0/3024980225 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0840a59c0 con 0x7fc08c0b4500 2023-10-16T08:38:21.454 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.449+0000 7fc08b7fe640 1 -- 172.21.15.86:0/3024980225 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0840a5d50 con 0x7fc08c0b4500 2023-10-16T08:38:21.454 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.449+0000 7fc092af2640 1 -- 172.21.15.86:0/3024980225 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc08c0b4500 msgr2=0x7fc08c0cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.454 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.449+0000 7fc092af2640 1 --2- 172.21.15.86:0/3024980225 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc08c0b4500 0x7fc08c0cd6d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc084009e50 tx=0x7fc08401c980 comp rx=0 tx=0).stop 2023-10-16T08:38:21.454 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.449+0000 7fc092af2640 1 -- 172.21.15.86:0/3024980225 shutdown_connections 2023-10-16T08:38:21.455 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.449+0000 7fc092af2640 1 --2- 172.21.15.86:0/3024980225 >> v2:172.21.15.40:3300/0 conn(0x7fc08c0cdc10 0x7fc08c06d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.455 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.449+0000 7fc092af2640 1 --2- 172.21.15.86:0/3024980225 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc08c0b4500 0x7fc08c0cd6d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.455 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.449+0000 7fc092af2640 1 -- 172.21.15.86:0/3024980225 >> 172.21.15.86:0/3024980225 conn(0x7fc08c0cf0f0 msgr2=0x7fc08c0cf500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:21.455 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.449+0000 7fc092af2640 1 -- 172.21.15.86:0/3024980225 shutdown_connections 2023-10-16T08:38:21.455 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.449+0000 7fc092af2640 1 -- 172.21.15.86:0/3024980225 wait complete. 2023-10-16T08:38:21.456 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc092af2640 1 Processor -- start 2023-10-16T08:38:21.456 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc092af2640 1 -- start start 2023-10-16T08:38:21.458 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc092af2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc08c06df00 0x7fc08c0c4c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.458 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc092af2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c0b4500 0x7fc08c0b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.458 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc092af2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc08c0cdc10 0x7fc08c0b2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.459 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc092af2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c140fd0 con 0x7fc08c0cdc10 2023-10-16T08:38:21.459 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc092af2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c154010 con 0x7fc08c0b4500 2023-10-16T08:38:21.459 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc092af2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc08c140cf0 con 0x7fc08c06df00 2023-10-16T08:38:21.459 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc09088e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc08c06df00 0x7fc08c0c4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.460 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc08bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c0b4500 0x7fc08c0b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.460 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc09088e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc08c06df00 0x7fc08c0c4c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:48702/0 (socket says 172.21.15.86:48702) 2023-10-16T08:38:21.460 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc09088e640 1 -- 172.21.15.86:0/548664571 learned_addr learned my addr 172.21.15.86:0/548664571 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-16T08:38:21.460 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc09108f640 1 --2- 172.21.15.86:0/548664571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc08c0cdc10 0x7fc08c0b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.460 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc09088e640 1 -- 172.21.15.86:0/548664571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c0b4500 msgr2=0x7fc08c0b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.460 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc09088e640 1 --2- 172.21.15.86:0/548664571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c0b4500 0x7fc08c0b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.461 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc09088e640 1 -- 172.21.15.86:0/548664571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc08c0cdc10 msgr2=0x7fc08c0b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.461 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc09088e640 1 --2- 172.21.15.86:0/548664571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc08c0cdc10 0x7fc08c0b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.461 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc09088e640 1 -- 172.21.15.86:0/548664571 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc084009b30 con 0x7fc08c06df00 2023-10-16T08:38:21.461 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc08bfff640 1 --2- 172.21.15.86:0/548664571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c0b4500 0x7fc08c0b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:38:21.461 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc09088e640 1 --2- 172.21.15.86:0/548664571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc08c06df00 0x7fc08c0c4c80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc08401a5b0 tx=0x7fc084019720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:21.462 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc0897fa640 1 -- 172.21.15.86:0/548664571 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0840b8940 con 0x7fc08c06df00 2023-10-16T08:38:21.462 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc0897fa640 1 -- 172.21.15.86:0/548664571 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0840b8aa0 con 0x7fc08c06df00 2023-10-16T08:38:21.462 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc0897fa640 1 -- 172.21.15.86:0/548664571 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0840b8e10 con 0x7fc08c06df00 2023-10-16T08:38:21.462 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc092af2640 1 -- 172.21.15.86:0/548664571 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc08c1542b0 con 0x7fc08c06df00 2023-10-16T08:38:21.462 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc092af2640 1 -- 172.21.15.86:0/548664571 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc08c154870 con 0x7fc08c06df00 2023-10-16T08:38:21.463 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc0897fa640 1 -- 172.21.15.86:0/548664571 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc084007770 con 0x7fc08c06df00 2023-10-16T08:38:21.463 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc0897fa640 1 -- 172.21.15.86:0/548664571 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7fc0840c1c80 con 0x7fc08c06df00 2023-10-16T08:38:21.463 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.453+0000 7fc092af2640 1 -- 172.21.15.86:0/548664571 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc08c04e9c0 con 0x7fc08c06df00 2023-10-16T08:38:21.463 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.457+0000 7fc0897fa640 1 -- 172.21.15.86:0/548664571 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fc0840dfb40 con 0x7fc08c06df00 2023-10-16T08:38:21.525 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.525+0000 7fc092af2640 1 -- 172.21.15.86:0/548664571 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32", "id": 2} v 0) v1 -- 0x7fc08c0cf800 con 0x7fc08c06df00 2023-10-16T08:38:21.532 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.529+0000 7fc0897fa640 1 -- 172.21.15.86:0/548664571 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fc0840ce030 con 0x7fc08c06df00 2023-10-16T08:38:21.533 INFO:teuthology.orchestra.run.smithi086.stdout:2 2023-10-16T08:38:21.534 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.533+0000 7fc092af2640 1 -- 172.21.15.86:0/548664571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc08c06df00 msgr2=0x7fc08c0c4c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.534 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.533+0000 7fc092af2640 1 --2- 172.21.15.86:0/548664571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc08c06df00 0x7fc08c0c4c80 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc08401a5b0 tx=0x7fc084019720 comp rx=0 tx=0).stop 2023-10-16T08:38:21.534 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.533+0000 7fc092af2640 1 -- 172.21.15.86:0/548664571 shutdown_connections 2023-10-16T08:38:21.535 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.533+0000 7fc092af2640 1 --2- 172.21.15.86:0/548664571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc08c0cdc10 0x7fc08c0b2340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.535 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.533+0000 7fc092af2640 1 --2- 172.21.15.86:0/548664571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc08c0b4500 0x7fc08c0b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.535 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.533+0000 7fc092af2640 1 --2- 172.21.15.86:0/548664571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc08c06df00 0x7fc08c0c4c80 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.535 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.533+0000 7fc092af2640 1 -- 172.21.15.86:0/548664571 >> 172.21.15.86:0/548664571 conn(0x7fc08c0cf0f0 msgr2=0x7fc08c0cb9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:21.535 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.533+0000 7fc092af2640 1 -- 172.21.15.86:0/548664571 shutdown_connections 2023-10-16T08:38:21.536 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.533+0000 7fc092af2640 1 -- 172.21.15.86:0/548664571 wait complete. 2023-10-16T08:38:21.550 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new f6ab290a-faf9-4699-93f1-0d2696a90b09 3 2023-10-16T08:38:21.668 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.665+0000 7f9bc9c44640 1 Processor -- start 2023-10-16T08:38:21.668 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.665+0000 7f9bc9c44640 1 -- start start 2023-10-16T08:38:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.665+0000 7f9bc9c44640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9bc415ad80 0x7f9bc406d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc9c44640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9bc406dba0 0x7f9bc40ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc9c44640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bc40cffd0 con 0x7f9bc40ceb00 2023-10-16T08:38:21.670 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc9c44640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bc406dfe0 con 0x7f9bc415ad80 2023-10-16T08:38:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc9c44640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc40cf310 con 0x7f9bc406dba0 2023-10-16T08:38:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc2ffd640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9bc406dba0 0x7f9bc40ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc3fff640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9bc40ceb00 0x7f9bc40cc3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44408/0 (socket says 172.21.15.86:44408) 2023-10-16T08:38:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc3fff640 1 -- 172.21.15.86:0/1382008428 learned_addr learned my addr 172.21.15.86:0/1382008428 (peer_addr_for_me v1:172.21.15.86:0/0) 2023-10-16T08:38:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc37fe640 1 --2- 172.21.15.86:0/1382008428 >> v2:172.21.15.40:3300/0 conn(0x7f9bc415ad80 0x7f9bc406d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.671 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc27fc640 1 -- 172.21.15.86:0/1382008428 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4141317644 0 0) 0x7f9bc40cffd0 con 0x7f9bc40ceb00 2023-10-16T08:38:21.672 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc27fc640 1 -- 172.21.15.86:0/1382008428 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bb0003530 con 0x7f9bc40ceb00 2023-10-16T08:38:21.672 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc2ffd640 1 -- 172.21.15.86:0/1382008428 >> v2:172.21.15.40:3300/0 conn(0x7f9bc415ad80 msgr2=0x7f9bc406d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:21.672 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc2ffd640 1 --2- 172.21.15.86:0/1382008428 >> v2:172.21.15.40:3300/0 conn(0x7f9bc415ad80 0x7f9bc406d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.672 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc2ffd640 1 -- 172.21.15.86:0/1382008428 >> v1:172.21.15.86:6789/0 conn(0x7f9bc40ceb00 legacy=0x7f9bc40cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.673 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc2ffd640 1 -- 172.21.15.86:0/1382008428 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc40cf5f0 con 0x7f9bc406dba0 2023-10-16T08:38:21.673 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc37fe640 1 --2- 172.21.15.86:0/1382008428 >> v2:172.21.15.40:3300/0 conn(0x7f9bc415ad80 0x7f9bc406d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:38:21.676 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc2ffd640 1 --2- 172.21.15.86:0/1382008428 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9bc406dba0 0x7f9bc40ce5c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9bac0097c0 tx=0x7f9bac01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d5d1bdf222fda11 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:21.676 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc27fc640 1 -- 172.21.15.86:0/1382008428 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac0ad660 con 0x7f9bc406dba0 2023-10-16T08:38:21.676 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc27fc640 1 -- 172.21.15.86:0/1382008428 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bac0ad7c0 con 0x7f9bc406dba0 2023-10-16T08:38:21.676 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc27fc640 1 -- 172.21.15.86:0/1382008428 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bac0adb50 con 0x7f9bc406dba0 2023-10-16T08:38:21.677 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc9c44640 1 -- 172.21.15.86:0/1382008428 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9bc406dba0 msgr2=0x7f9bc40ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.677 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc9c44640 1 --2- 172.21.15.86:0/1382008428 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9bc406dba0 0x7f9bc40ce5c0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9bac0097c0 tx=0x7f9bac01c450 comp rx=0 tx=0).stop 2023-10-16T08:38:21.677 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc9c44640 1 -- 172.21.15.86:0/1382008428 shutdown_connections 2023-10-16T08:38:21.677 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc9c44640 1 --2- 172.21.15.86:0/1382008428 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9bc406dba0 0x7f9bc40ce5c0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.677 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc9c44640 1 --2- 172.21.15.86:0/1382008428 >> v2:172.21.15.40:3300/0 conn(0x7f9bc415ad80 0x7f9bc406d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.678 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc9c44640 1 -- 172.21.15.86:0/1382008428 >> 172.21.15.86:0/1382008428 conn(0x7f9bc4071580 msgr2=0x7f9bc4071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:21.678 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc9c44640 1 -- 172.21.15.86:0/1382008428 shutdown_connections 2023-10-16T08:38:21.678 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.669+0000 7f9bc9c44640 1 -- 172.21.15.86:0/1382008428 wait complete. 2023-10-16T08:38:21.678 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc9c44640 1 Processor -- start 2023-10-16T08:38:21.678 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc9c44640 1 -- start start 2023-10-16T08:38:21.679 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc9c44640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bc406dba0 0x7f9bc4152e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.679 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc9c44640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bc40ceb00 0x7f9bc4159a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.679 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc9c44640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9bc415ad80 0x7f9bc4147d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.679 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc9c44640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc4152060 con 0x7f9bc40ceb00 2023-10-16T08:38:21.679 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc9c44640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc4149e30 con 0x7f9bc406dba0 2023-10-16T08:38:21.679 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc9c44640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bc4151450 con 0x7f9bc415ad80 2023-10-16T08:38:21.680 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bc406dba0 0x7f9bc4152e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.680 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc3fff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9bc415ad80 0x7f9bc4147d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.680 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bc406dba0 0x7f9bc4152e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59198/0 (socket says 172.21.15.86:59198) 2023-10-16T08:38:21.680 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc37fe640 1 -- 172.21.15.86:0/2536417739 learned_addr learned my addr 172.21.15.86:0/2536417739 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-16T08:38:21.680 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc2ffd640 1 --2- 172.21.15.86:0/2536417739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bc40ceb00 0x7f9bc4159a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc3fff640 1 -- 172.21.15.86:0/2536417739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bc406dba0 msgr2=0x7f9bc4152e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc3fff640 1 --2- 172.21.15.86:0/2536417739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bc406dba0 0x7f9bc4152e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc3fff640 1 -- 172.21.15.86:0/2536417739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bc40ceb00 msgr2=0x7f9bc4159a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc3fff640 1 --2- 172.21.15.86:0/2536417739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bc40ceb00 0x7f9bc4159a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc3fff640 1 -- 172.21.15.86:0/2536417739 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bac009420 con 0x7f9bc415ad80 2023-10-16T08:38:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc37fe640 1 --2- 172.21.15.86:0/2536417739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bc406dba0 0x7f9bc4152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:38:21.682 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc3fff640 1 --2- 172.21.15.86:0/2536417739 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9bc415ad80 0x7f9bc4147d80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400ca80 tx=0x7f9bb400cf40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:21.682 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bbbfff640 1 -- 172.21.15.86:0/2536417739 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb400ac30 con 0x7f9bc415ad80 2023-10-16T08:38:21.682 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bbbfff640 1 -- 172.21.15.86:0/2536417739 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bb400ad90 con 0x7f9bc415ad80 2023-10-16T08:38:21.682 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bbbfff640 1 -- 172.21.15.86:0/2536417739 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bb4012600 con 0x7f9bc415ad80 2023-10-16T08:38:21.682 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc9c44640 1 -- 172.21.15.86:0/2536417739 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bc4151730 con 0x7f9bc415ad80 2023-10-16T08:38:21.683 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc9c44640 1 -- 172.21.15.86:0/2536417739 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bc4151cf0 con 0x7f9bc415ad80 2023-10-16T08:38:21.683 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bbbfff640 1 -- 172.21.15.86:0/2536417739 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9bb4012a60 con 0x7f9bc415ad80 2023-10-16T08:38:21.683 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bbbfff640 1 -- 172.21.15.86:0/2536417739 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f9bb4012d10 con 0x7f9bc415ad80 2023-10-16T08:38:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.673+0000 7f9bc9c44640 1 -- 172.21.15.86:0/2536417739 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bc404e9c0 con 0x7f9bc415ad80 2023-10-16T08:38:21.684 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.677+0000 7f9bbbfff640 1 -- 172.21.15.86:0/2536417739 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f9bb4021060 con 0x7f9bc415ad80 2023-10-16T08:38:21.746 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.745+0000 7f9bc9c44640 1 -- 172.21.15.86:0/2536417739 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd new", "uuid": "f6ab290a-faf9-4699-93f1-0d2696a90b09", "id": 3} v 0) v1 -- 0x7f9bc40b1e60 con 0x7f9bc415ad80 2023-10-16T08:38:21.754 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.753+0000 7f9bbbfff640 1 -- 172.21.15.86:0/2536417739 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "f6ab290a-faf9-4699-93f1-0d2696a90b09", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f9bb4017070 con 0x7f9bc415ad80 2023-10-16T08:38:21.755 INFO:teuthology.orchestra.run.smithi086.stdout:3 2023-10-16T08:38:21.756 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.753+0000 7f9bc9c44640 1 -- 172.21.15.86:0/2536417739 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9bc415ad80 msgr2=0x7f9bc4147d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.756 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.753+0000 7f9bc9c44640 1 --2- 172.21.15.86:0/2536417739 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9bc415ad80 0x7f9bc4147d80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400ca80 tx=0x7f9bb400cf40 comp rx=0 tx=0).stop 2023-10-16T08:38:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.753+0000 7f9bc9c44640 1 -- 172.21.15.86:0/2536417739 shutdown_connections 2023-10-16T08:38:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.753+0000 7f9bc9c44640 1 --2- 172.21.15.86:0/2536417739 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9bc415ad80 0x7f9bc4147d80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.753+0000 7f9bc9c44640 1 --2- 172.21.15.86:0/2536417739 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9bc40ceb00 0x7f9bc4159a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.758 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.753+0000 7f9bc9c44640 1 --2- 172.21.15.86:0/2536417739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9bc406dba0 0x7f9bc4152e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.758 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.753+0000 7f9bc9c44640 1 -- 172.21.15.86:0/2536417739 >> 172.21.15.86:0/2536417739 conn(0x7f9bc4071580 msgr2=0x7f9bc40ca2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:21.759 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.753+0000 7f9bc9c44640 1 -- 172.21.15.86:0/2536417739 shutdown_connections 2023-10-16T08:38:21.759 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.753+0000 7f9bc9c44640 1 -- 172.21.15.86:0/2536417739 wait complete. 2023-10-16T08:38:21.771 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 89381ba9-8997-4473-a63f-c142d2231ece 4 2023-10-16T08:38:21.885 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:21.881+0000 7f284cdce180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T08:38:21.886 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.885+0000 7fb27f323640 1 Processor -- start 2023-10-16T08:38:21.887 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.885+0000 7fb27f323640 1 -- start start 2023-10-16T08:38:21.888 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.885+0000 7fb27f323640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb2780cde70 0x7fb27806d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.889 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.885+0000 7fb27f323640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2780b19e0 0x7fb2781594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.889 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.885+0000 7fb27f323640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb27806c580 con 0x7fb2780ce250 2023-10-16T08:38:21.889 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.885+0000 7fb27f323640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb27806c860 con 0x7fb2780cde70 2023-10-16T08:38:21.889 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.885+0000 7fb27f323640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb27806cb40 con 0x7fb2780b19e0 2023-10-16T08:38:21.890 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.885+0000 7fb27d8c0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2780b19e0 0x7fb2781594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.890 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.885+0000 7fb27d8c0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2780b19e0 0x7fb2781594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:48734/0 (socket says 172.21.15.86:48734) 2023-10-16T08:38:21.890 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.885+0000 7fb27d8c0640 1 -- 172.21.15.86:0/495988806 learned_addr learned my addr 172.21.15.86:0/495988806 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-16T08:38:21.890 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.885+0000 7fb27d0bf640 1 --2- 172.21.15.86:0/495988806 >> v2:172.21.15.40:3300/0 conn(0x7fb2780cde70 0x7fb27806d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.891 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.885+0000 7fb26ffff640 1 -- 172.21.15.86:0/495988806 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 817400358 0 0) 0x7fb27806c580 con 0x7fb2780ce250 2023-10-16T08:38:21.891 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.885+0000 7fb26ffff640 1 -- 172.21.15.86:0/495988806 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb264003530 con 0x7fb2780ce250 2023-10-16T08:38:21.891 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.885+0000 7fb27d8c0640 1 -- 172.21.15.86:0/495988806 >> v2:172.21.15.40:3300/0 conn(0x7fb2780cde70 msgr2=0x7fb27806d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:21.891 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.885+0000 7fb27d8c0640 1 --2- 172.21.15.86:0/495988806 >> v2:172.21.15.40:3300/0 conn(0x7fb2780cde70 0x7fb27806d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.891 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.885+0000 7fb27d8c0640 1 -- 172.21.15.86:0/495988806 >> v1:172.21.15.86:6789/0 conn(0x7fb2780ce250 legacy=0x7fb2780b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.892 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.885+0000 7fb27d8c0640 1 -- 172.21.15.86:0/495988806 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb27806ce20 con 0x7fb2780b19e0 2023-10-16T08:38:21.894 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.889+0000 7fb27d8c0640 1 --2- 172.21.15.86:0/495988806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2780b19e0 0x7fb2781594a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb270009e50 tx=0x7fb27001ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75d1214f1280ceee server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:21.895 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.889+0000 7fb26ffff640 1 -- 172.21.15.86:0/495988806 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2700a5840 con 0x7fb2780b19e0 2023-10-16T08:38:21.895 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.889+0000 7fb26ffff640 1 -- 172.21.15.86:0/495988806 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2700a59a0 con 0x7fb2780b19e0 2023-10-16T08:38:21.895 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.889+0000 7fb26ffff640 1 -- 172.21.15.86:0/495988806 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2700a5d30 con 0x7fb2780b19e0 2023-10-16T08:38:21.895 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.889+0000 7fb27f323640 1 -- 172.21.15.86:0/495988806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2780b19e0 msgr2=0x7fb2781594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.895 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.889+0000 7fb27f323640 1 --2- 172.21.15.86:0/495988806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2780b19e0 0x7fb2781594a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fb270009e50 tx=0x7fb27001ca10 comp rx=0 tx=0).stop 2023-10-16T08:38:21.896 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.889+0000 7fb27f323640 1 -- 172.21.15.86:0/495988806 shutdown_connections 2023-10-16T08:38:21.896 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.889+0000 7fb27f323640 1 --2- 172.21.15.86:0/495988806 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2780b19e0 0x7fb2781594a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.896 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.889+0000 7fb27f323640 1 --2- 172.21.15.86:0/495988806 >> v2:172.21.15.40:3300/0 conn(0x7fb2780cde70 0x7fb27806d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.896 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.889+0000 7fb27f323640 1 -- 172.21.15.86:0/495988806 >> 172.21.15.86:0/495988806 conn(0x7fb2780cc2d0 msgr2=0x7fb2780cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:21.896 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.889+0000 7fb27f323640 1 -- 172.21.15.86:0/495988806 shutdown_connections 2023-10-16T08:38:21.897 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.889+0000 7fb27f323640 1 -- 172.21.15.86:0/495988806 wait complete. 2023-10-16T08:38:21.897 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.889+0000 7fb27f323640 1 Processor -- start 2023-10-16T08:38:21.897 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.889+0000 7fb27f323640 1 -- start start 2023-10-16T08:38:21.897 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27f323640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2780b19e0 0x7fb2780c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.898 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27f323640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2780cde70 0x7fb2780c54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.898 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27f323640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2780ce250 0x7fb278150960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:21.898 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27f323640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2780c9910 con 0x7fb2780cde70 2023-10-16T08:38:21.898 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27f323640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2780c84a0 con 0x7fb2780ce250 2023-10-16T08:38:21.899 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27f323640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2780c8780 con 0x7fb2780b19e0 2023-10-16T08:38:21.899 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27d0bf640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2780b19e0 0x7fb2780c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.899 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27d0bf640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2780b19e0 0x7fb2780c9330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:48738/0 (socket says 172.21.15.86:48738) 2023-10-16T08:38:21.899 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27d0bf640 1 -- 172.21.15.86:0/2783448383 learned_addr learned my addr 172.21.15.86:0/2783448383 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-16T08:38:21.899 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27d8c0640 1 --2- 172.21.15.86:0/2783448383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2780ce250 0x7fb278150960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.899 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27c8be640 1 --2- 172.21.15.86:0/2783448383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2780cde70 0x7fb2780c54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:21.900 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27d0bf640 1 -- 172.21.15.86:0/2783448383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2780ce250 msgr2=0x7fb278150960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.900 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27d0bf640 1 --2- 172.21.15.86:0/2783448383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2780ce250 0x7fb278150960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.900 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27d0bf640 1 -- 172.21.15.86:0/2783448383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2780cde70 msgr2=0x7fb2780c54f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.900 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27d0bf640 1 --2- 172.21.15.86:0/2783448383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2780cde70 0x7fb2780c54f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.900 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27d0bf640 1 -- 172.21.15.86:0/2783448383 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb270009b30 con 0x7fb2780b19e0 2023-10-16T08:38:21.901 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27d8c0640 1 --2- 172.21.15.86:0/2783448383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2780ce250 0x7fb278150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:38:21.901 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27d0bf640 1 --2- 172.21.15.86:0/2783448383 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2780b19e0 0x7fb2780c9330 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb27400d5f0 tx=0x7fb27400dab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:21.901 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb26dffb640 1 -- 172.21.15.86:0/2783448383 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2740125f0 con 0x7fb2780b19e0 2023-10-16T08:38:21.901 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb26dffb640 1 -- 172.21.15.86:0/2783448383 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb274012750 con 0x7fb2780b19e0 2023-10-16T08:38:21.901 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27f323640 1 -- 172.21.15.86:0/2783448383 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2780c8a60 con 0x7fb2780b19e0 2023-10-16T08:38:21.902 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb26dffb640 1 -- 172.21.15.86:0/2783448383 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb274012ae0 con 0x7fb2780b19e0 2023-10-16T08:38:21.902 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27f323640 1 -- 172.21.15.86:0/2783448383 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb27815b260 con 0x7fb2780b19e0 2023-10-16T08:38:21.902 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb26dffb640 1 -- 172.21.15.86:0/2783448383 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fb27401e460 con 0x7fb2780b19e0 2023-10-16T08:38:21.902 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb26dffb640 1 -- 172.21.15.86:0/2783448383 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fb274014990 con 0x7fb2780b19e0 2023-10-16T08:38:21.903 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.893+0000 7fb27f323640 1 -- 172.21.15.86:0/2783448383 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb27804e9c0 con 0x7fb2780b19e0 2023-10-16T08:38:21.903 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.897+0000 7fb26dffb640 1 -- 172.21.15.86:0/2783448383 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7fb274014c70 con 0x7fb2780b19e0 2023-10-16T08:38:21.945 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:21.941+0000 7f284cdce180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T08:38:21.965 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.965+0000 7fb27f323640 1 -- 172.21.15.86:0/2783448383 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd new", "uuid": "89381ba9-8997-4473-a63f-c142d2231ece", "id": 4} v 0) v1 -- 0x7fb2780b57d0 con 0x7fb2780b19e0 2023-10-16T08:38:21.972 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.969+0000 7fb26dffb640 1 -- 172.21.15.86:0/2783448383 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "89381ba9-8997-4473-a63f-c142d2231ece", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fb274014480 con 0x7fb2780b19e0 2023-10-16T08:38:21.972 INFO:teuthology.orchestra.run.smithi086.stdout:4 2023-10-16T08:38:21.973 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.969+0000 7fb27f323640 1 -- 172.21.15.86:0/2783448383 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2780b19e0 msgr2=0x7fb2780c9330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:21.973 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.969+0000 7fb27f323640 1 --2- 172.21.15.86:0/2783448383 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2780b19e0 0x7fb2780c9330 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fb27400d5f0 tx=0x7fb27400dab0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.974 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.973+0000 7fb27f323640 1 -- 172.21.15.86:0/2783448383 shutdown_connections 2023-10-16T08:38:21.974 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.973+0000 7fb27f323640 1 --2- 172.21.15.86:0/2783448383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2780ce250 0x7fb278150960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.974 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.973+0000 7fb27f323640 1 --2- 172.21.15.86:0/2783448383 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2780cde70 0x7fb2780c54f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.974 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.973+0000 7fb27f323640 1 --2- 172.21.15.86:0/2783448383 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2780b19e0 0x7fb2780c9330 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:21.975 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.973+0000 7fb27f323640 1 -- 172.21.15.86:0/2783448383 >> 172.21.15.86:0/2783448383 conn(0x7fb2780cc2d0 msgr2=0x7fb2780cb330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:21.975 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.973+0000 7fb27f323640 1 -- 172.21.15.86:0/2783448383 shutdown_connections 2023-10-16T08:38:21.975 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:21.973+0000 7fb27f323640 1 -- 172.21.15.86:0/2783448383 wait complete. 2023-10-16T08:38:21.989 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 0467d942-124c-414c-b589-a79570b74d88 5 2023-10-16T08:38:22.004 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:22.001+0000 7f284cdce180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T08:38:22.083 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:22.081+0000 7f284cdce180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T08:38:22.094 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:22.089+0000 7f5901f19180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-16T08:38:22.103 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.101+0000 7f6481983640 1 Processor -- start 2023-10-16T08:38:22.103 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.101+0000 7f6481983640 1 -- start start 2023-10-16T08:38:22.104 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.101+0000 7f6481983640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f647c15ad00 0x7f647c0cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.105 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.101+0000 7f6481983640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f647c0cb800 0x7f647c065a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.105 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.101+0000 7f6481983640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f647c065fc0 0x7f647c06bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.105 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.101+0000 7f6481983640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f647c0cbc80 con 0x7f647c15ad00 2023-10-16T08:38:22.105 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.101+0000 7f6481983640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f647c066450 con 0x7f647c0cb800 2023-10-16T08:38:22.105 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.101+0000 7f6481983640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f647c0b12d0 con 0x7f647c065fc0 2023-10-16T08:38:22.105 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.101+0000 7f647a7fc640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f647c0cb800 0x7f647c065a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.106 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.101+0000 7f647b7fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f647c065fc0 0x7f647c06bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.106 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.101+0000 7f647a7fc640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f647c0cb800 0x7f647c065a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59210/0 (socket says 172.21.15.86:59210) 2023-10-16T08:38:22.106 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.101+0000 7f647a7fc640 1 -- 172.21.15.86:0/384612678 learned_addr learned my addr 172.21.15.86:0/384612678 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-16T08:38:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.101+0000 7f647affd640 1 --2- 172.21.15.86:0/384612678 >> v2:172.21.15.40:3300/0 conn(0x7f647c15ad00 0x7f647c0cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.101+0000 7f647a7fc640 1 -- 172.21.15.86:0/384612678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f647c065fc0 msgr2=0x7f647c06bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.101+0000 7f647a7fc640 1 --2- 172.21.15.86:0/384612678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f647c065fc0 0x7f647c06bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.101+0000 7f647a7fc640 1 -- 172.21.15.86:0/384612678 >> v2:172.21.15.40:3300/0 conn(0x7f647c15ad00 msgr2=0x7f647c0cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.101+0000 7f647a7fc640 1 --2- 172.21.15.86:0/384612678 >> v2:172.21.15.40:3300/0 conn(0x7f647c15ad00 0x7f647c0cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.101+0000 7f647a7fc640 1 -- 172.21.15.86:0/384612678 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f647c0b1b30 con 0x7f647c0cb800 2023-10-16T08:38:22.108 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.101+0000 7f647b7fe640 1 --2- 172.21.15.86:0/384612678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f647c065fc0 0x7f647c06bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:38:22.108 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.101+0000 7f647affd640 1 --2- 172.21.15.86:0/384612678 >> v2:172.21.15.40:3300/0 conn(0x7f647c15ad00 0x7f647c0cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:22.111 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.105+0000 7f647a7fc640 1 --2- 172.21.15.86:0/384612678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f647c0cb800 0x7f647c065a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f647c0cf790 tx=0x7f646401c240 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e202ecc45404769f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:22.111 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.105+0000 7f6479ffb640 1 -- 172.21.15.86:0/384612678 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64640ab600 con 0x7f647c0cb800 2023-10-16T08:38:22.111 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.105+0000 7f6479ffb640 1 -- 172.21.15.86:0/384612678 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64640a7050 con 0x7f647c0cb800 2023-10-16T08:38:22.112 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.105+0000 7f6479ffb640 1 -- 172.21.15.86:0/384612678 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64640ab980 con 0x7f647c0cb800 2023-10-16T08:38:22.112 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.105+0000 7f6481983640 1 -- 172.21.15.86:0/384612678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f647c0cb800 msgr2=0x7f647c065a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:22.112 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.105+0000 7f6481983640 1 --2- 172.21.15.86:0/384612678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f647c0cb800 0x7f647c065a80 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f647c0cf790 tx=0x7f646401c240 comp rx=0 tx=0).stop 2023-10-16T08:38:22.112 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.105+0000 7f6481983640 1 -- 172.21.15.86:0/384612678 shutdown_connections 2023-10-16T08:38:22.112 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.105+0000 7f6481983640 1 --2- 172.21.15.86:0/384612678 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f647c065fc0 0x7f647c06bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.112 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.105+0000 7f6481983640 1 --2- 172.21.15.86:0/384612678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f647c0cb800 0x7f647c065a80 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.113 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.105+0000 7f6481983640 1 --2- 172.21.15.86:0/384612678 >> v2:172.21.15.40:3300/0 conn(0x7f647c15ad00 0x7f647c0cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.113 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.105+0000 7f6481983640 1 -- 172.21.15.86:0/384612678 >> 172.21.15.86:0/384612678 conn(0x7f647c0cc2d0 msgr2=0x7f647c0cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:22.113 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.105+0000 7f6481983640 1 -- 172.21.15.86:0/384612678 shutdown_connections 2023-10-16T08:38:22.113 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.105+0000 7f6481983640 1 -- 172.21.15.86:0/384612678 wait complete. 2023-10-16T08:38:22.113 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f6481983640 1 Processor -- start 2023-10-16T08:38:22.114 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f6481983640 1 -- start start 2023-10-16T08:38:22.114 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f6481983640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f647c065fc0 0x7f647c0b3520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.114 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f6481983640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f647c0cb800 0x7f647c156dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.114 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f6481983640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f647c15ad00 0x7f647c157310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.114 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f6481983640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c153290 con 0x7f647c065fc0 2023-10-16T08:38:22.115 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f6481983640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f647c0cf3e0 con 0x7f647c15ad00 2023-10-16T08:38:22.115 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f6481983640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f647c0ce940 con 0x7f647c0cb800 2023-10-16T08:38:22.115 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f647b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f647c15ad00 0x7f647c157310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.115 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f647a7fc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f647c0cb800 0x7f647c156dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.115 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f647a7fc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f647c0cb800 0x7f647c156dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:48750/0 (socket says 172.21.15.86:48750) 2023-10-16T08:38:22.116 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f647b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f647c15ad00 0x7f647c157310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59216/0 (socket says 172.21.15.86:59216) 2023-10-16T08:38:22.116 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f647a7fc640 1 -- 172.21.15.86:0/1508516989 learned_addr learned my addr 172.21.15.86:0/1508516989 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-16T08:38:22.116 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f647affd640 1 --2- 172.21.15.86:0/1508516989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f647c065fc0 0x7f647c0b3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.116 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f647a7fc640 1 -- 172.21.15.86:0/1508516989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f647c15ad00 msgr2=0x7f647c157310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f647a7fc640 1 --2- 172.21.15.86:0/1508516989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f647c15ad00 0x7f647c157310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f647a7fc640 1 -- 172.21.15.86:0/1508516989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f647c065fc0 msgr2=0x7f647c0b3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f647a7fc640 1 --2- 172.21.15.86:0/1508516989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f647c065fc0 0x7f647c0b3520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f647a7fc640 1 -- 172.21.15.86:0/1508516989 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6474009a70 con 0x7f647c0cb800 2023-10-16T08:38:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f647a7fc640 1 --2- 172.21.15.86:0/1508516989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f647c0cb800 0x7f647c156dd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f646401cae0 tx=0x7f6464018fe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f6473fff640 1 -- 172.21.15.86:0/1508516989 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64640ba070 con 0x7f647c0cb800 2023-10-16T08:38:22.118 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f6473fff640 1 -- 172.21.15.86:0/1508516989 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64640abda0 con 0x7f647c0cb800 2023-10-16T08:38:22.118 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f6481983640 1 -- 172.21.15.86:0/1508516989 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64640094d0 con 0x7f647c0cb800 2023-10-16T08:38:22.118 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f6473fff640 1 -- 172.21.15.86:0/1508516989 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64640b66a0 con 0x7f647c0cb800 2023-10-16T08:38:22.118 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f6481983640 1 -- 172.21.15.86:0/1508516989 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f647c06ded0 con 0x7f647c0cb800 2023-10-16T08:38:22.118 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f6473fff640 1 -- 172.21.15.86:0/1508516989 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f64640b2040 con 0x7f647c0cb800 2023-10-16T08:38:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f6473fff640 1 -- 172.21.15.86:0/1508516989 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f64640b4c30 con 0x7f647c0cb800 2023-10-16T08:38:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.109+0000 7f6481983640 1 -- 172.21.15.86:0/1508516989 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f647c04e9c0 con 0x7f647c0cb800 2023-10-16T08:38:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.113+0000 7f6473fff640 1 -- 172.21.15.86:0/1508516989 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f64640b0070 con 0x7f647c0cb800 2023-10-16T08:38:22.138 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:22.137+0000 7f284cdce180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T08:38:22.150 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:22.145+0000 7f5901f19180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-16T08:38:22.181 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.181+0000 7f6481983640 1 -- 172.21.15.86:0/1508516989 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0467d942-124c-414c-b589-a79570b74d88", "id": 5} v 0) v1 -- 0x7f647c152cd0 con 0x7f647c0cb800 2023-10-16T08:38:22.188 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.185+0000 7f6473fff640 1 -- 172.21.15.86:0/1508516989 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0467d942-124c-414c-b589-a79570b74d88", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f64640b4df0 con 0x7f647c0cb800 2023-10-16T08:38:22.188 INFO:teuthology.orchestra.run.smithi086.stdout:5 2023-10-16T08:38:22.190 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.189+0000 7f6481983640 1 -- 172.21.15.86:0/1508516989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f647c0cb800 msgr2=0x7f647c156dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:22.190 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.189+0000 7f6481983640 1 --2- 172.21.15.86:0/1508516989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f647c0cb800 0x7f647c156dd0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f646401cae0 tx=0x7f6464018fe0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.190 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.189+0000 7f6481983640 1 -- 172.21.15.86:0/1508516989 shutdown_connections 2023-10-16T08:38:22.191 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.189+0000 7f6481983640 1 --2- 172.21.15.86:0/1508516989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f647c15ad00 0x7f647c157310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.191 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.189+0000 7f6481983640 1 --2- 172.21.15.86:0/1508516989 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f647c0cb800 0x7f647c156dd0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.191 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.189+0000 7f6481983640 1 --2- 172.21.15.86:0/1508516989 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f647c065fc0 0x7f647c0b3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.191 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.189+0000 7f6481983640 1 -- 172.21.15.86:0/1508516989 >> 172.21.15.86:0/1508516989 conn(0x7f647c0cc2d0 msgr2=0x7f647c0b04d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:22.191 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.189+0000 7f6481983640 1 -- 172.21.15.86:0/1508516989 shutdown_connections 2023-10-16T08:38:22.191 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.189+0000 7f6481983640 1 -- 172.21.15.86:0/1508516989 wait complete. 2023-10-16T08:38:22.205 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:22.201+0000 7f5901f19180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-16T08:38:22.206 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 3ffc263f-93b2-436d-8106-a45438435c39 6 2023-10-16T08:38:22.218 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:22.217+0000 7f284cdce180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T08:38:22.272 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:22.269+0000 7f284cdce180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T08:38:22.285 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:22.281+0000 7f5901f19180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-16T08:38:22.336 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.333+0000 7ffbd80eb640 1 Processor -- start 2023-10-16T08:38:22.337 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.333+0000 7ffbd80eb640 1 -- start start 2023-10-16T08:38:22.338 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.337+0000 7ffbd80eb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffbd0066190 0x7ffbd00710c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.338 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.337+0000 7ffbd80eb640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ffbd01614c0 0x7ffbd00c9ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.339 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.337+0000 7ffbd80eb640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbd00704e0 con 0x7ffbd01602a0 2023-10-16T08:38:22.339 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.337+0000 7ffbd80eb640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbd00af170 con 0x7ffbd01614c0 2023-10-16T08:38:22.339 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.337+0000 7ffbd80eb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd0070a10 con 0x7ffbd0066190 2023-10-16T08:38:22.339 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.337+0000 7ffbd5686640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffbd0066190 0x7ffbd00710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.339 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.337+0000 7ffbd5686640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffbd0066190 0x7ffbd00710c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:48754/0 (socket says 172.21.15.86:48754) 2023-10-16T08:38:22.339 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.337+0000 7ffbd5686640 1 -- 172.21.15.86:0/1275413003 learned_addr learned my addr 172.21.15.86:0/1275413003 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-16T08:38:22.340 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.337+0000 7ffbd6688640 1 --2- 172.21.15.86:0/1275413003 >> v2:172.21.15.40:3300/0 conn(0x7ffbd01614c0 0x7ffbd00c9ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.340 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.337+0000 7ffbd5686640 1 -- 172.21.15.86:0/1275413003 >> v2:172.21.15.40:3300/0 conn(0x7ffbd01614c0 msgr2=0x7ffbd00c9ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:22.340 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.337+0000 7ffbd5686640 1 --2- 172.21.15.86:0/1275413003 >> v2:172.21.15.40:3300/0 conn(0x7ffbd01614c0 0x7ffbd00c9ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.340 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.337+0000 7ffbd5686640 1 -- 172.21.15.86:0/1275413003 >> v1:172.21.15.40:6789/0 conn(0x7ffbd01602a0 legacy=0x7ffbd0065a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:22.341 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.337+0000 7ffbd5686640 1 -- 172.21.15.86:0/1275413003 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd0161030 con 0x7ffbd0066190 2023-10-16T08:38:22.341 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:22.337+0000 7f5901f19180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-16T08:38:22.343 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.337+0000 7ffbd5686640 1 --2- 172.21.15.86:0/1275413003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffbd0066190 0x7ffbd00710c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009870 tx=0x7ffbc001c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f56c36b4d14a3d33 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:22.343 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.337+0000 7ffbd4e85640 1 -- 172.21.15.86:0/1275413003 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc00a7070 con 0x7ffbd0066190 2023-10-16T08:38:22.343 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.337+0000 7ffbd4e85640 1 -- 172.21.15.86:0/1275413003 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbc00ab430 con 0x7ffbd0066190 2023-10-16T08:38:22.343 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.337+0000 7ffbd4e85640 1 -- 172.21.15.86:0/1275413003 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbc00ab7c0 con 0x7ffbd0066190 2023-10-16T08:38:22.345 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd80eb640 1 -- 172.21.15.86:0/1275413003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffbd0066190 msgr2=0x7ffbd00710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:22.345 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd80eb640 1 --2- 172.21.15.86:0/1275413003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffbd0066190 0x7ffbd00710c0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7ffbc0009870 tx=0x7ffbc001c540 comp rx=0 tx=0).stop 2023-10-16T08:38:22.345 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd80eb640 1 -- 172.21.15.86:0/1275413003 shutdown_connections 2023-10-16T08:38:22.346 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd80eb640 1 --2- 172.21.15.86:0/1275413003 >> v2:172.21.15.40:3300/0 conn(0x7ffbd01614c0 0x7ffbd00c9ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.346 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd80eb640 1 --2- 172.21.15.86:0/1275413003 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffbd0066190 0x7ffbd00710c0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.346 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd80eb640 1 -- 172.21.15.86:0/1275413003 >> 172.21.15.86:0/1275413003 conn(0x7ffbd00cc2d0 msgr2=0x7ffbd00cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:22.346 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd80eb640 1 -- 172.21.15.86:0/1275413003 shutdown_connections 2023-10-16T08:38:22.347 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd80eb640 1 -- 172.21.15.86:0/1275413003 wait complete. 2023-10-16T08:38:22.347 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd80eb640 1 Processor -- start 2023-10-16T08:38:22.349 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd80eb640 1 -- start start 2023-10-16T08:38:22.349 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd80eb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffbd0066190 0x7ffbd0156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.349 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd80eb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd01602a0 0x7ffbd0157d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.349 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd80eb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbd01614c0 0x7ffbd01548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.350 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd80eb640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd01567c0 con 0x7ffbd01614c0 2023-10-16T08:38:22.350 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd80eb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd0156c70 con 0x7ffbd01602a0 2023-10-16T08:38:22.350 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd80eb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbd0156f50 con 0x7ffbd0066190 2023-10-16T08:38:22.350 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd5e87640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffbd0066190 0x7ffbd0156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.351 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd5686640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd01602a0 0x7ffbd0157d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.351 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd5e87640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffbd0066190 0x7ffbd0156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:48756/0 (socket says 172.21.15.86:48756) 2023-10-16T08:38:22.351 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd5e87640 1 -- 172.21.15.86:0/2664341499 learned_addr learned my addr 172.21.15.86:0/2664341499 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-16T08:38:22.351 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd6688640 1 --2- 172.21.15.86:0/2664341499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbd01614c0 0x7ffbd01548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.351 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd5e87640 1 -- 172.21.15.86:0/2664341499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd01602a0 msgr2=0x7ffbd0157d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:22.352 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd5e87640 1 --2- 172.21.15.86:0/2664341499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd01602a0 0x7ffbd0157d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.352 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd5e87640 1 -- 172.21.15.86:0/2664341499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbd01614c0 msgr2=0x7ffbd01548a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:22.352 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd5e87640 1 --2- 172.21.15.86:0/2664341499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbd01614c0 0x7ffbd01548a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.352 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd5e87640 1 -- 172.21.15.86:0/2664341499 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc00094d0 con 0x7ffbd0066190 2023-10-16T08:38:22.352 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd5686640 1 --2- 172.21.15.86:0/2664341499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd01602a0 0x7ffbd0157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:38:22.353 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd5e87640 1 --2- 172.21.15.86:0/2664341499 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffbd0066190 0x7ffbd0156280 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00c500 tx=0x7ffbcc00c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:22.353 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbc67fc640 1 -- 172.21.15.86:0/2664341499 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc005220 con 0x7ffbd0066190 2023-10-16T08:38:22.353 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbc67fc640 1 -- 172.21.15.86:0/2664341499 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbcc005380 con 0x7ffbd0066190 2023-10-16T08:38:22.353 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd80eb640 1 -- 172.21.15.86:0/2664341499 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbd0157230 con 0x7ffbd0066190 2023-10-16T08:38:22.353 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbc67fc640 1 -- 172.21.15.86:0/2664341499 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbcc007270 con 0x7ffbd0066190 2023-10-16T08:38:22.353 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbd80eb640 1 -- 172.21.15.86:0/2664341499 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbd01577f0 con 0x7ffbd0066190 2023-10-16T08:38:22.354 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbc67fc640 1 -- 172.21.15.86:0/2664341499 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7ffbcc0067e0 con 0x7ffbd0066190 2023-10-16T08:38:22.354 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.341+0000 7ffbc67fc640 1 -- 172.21.15.86:0/2664341499 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7ffbcc01b400 con 0x7ffbd0066190 2023-10-16T08:38:22.354 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.345+0000 7ffbd80eb640 1 -- 172.21.15.86:0/2664341499 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbd01579e0 con 0x7ffbd0066190 2023-10-16T08:38:22.354 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.345+0000 7ffbc67fc640 1 -- 172.21.15.86:0/2664341499 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7ffbcc016020 con 0x7ffbd0066190 2023-10-16T08:38:22.414 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.413+0000 7ffbd80eb640 1 -- 172.21.15.86:0/2664341499 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd new", "uuid": "3ffc263f-93b2-436d-8106-a45438435c39", "id": 6} v 0) v1 -- 0x7ffbd004e9c0 con 0x7ffbd0066190 2023-10-16T08:38:22.415 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:22.413+0000 7f284cdce180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T08:38:22.420 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:22.417+0000 7f5901f19180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-16T08:38:22.423 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.421+0000 7ffbc67fc640 1 -- 172.21.15.86:0/2664341499 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "3ffc263f-93b2-436d-8106-a45438435c39", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7ffbcc02a0b0 con 0x7ffbd0066190 2023-10-16T08:38:22.424 INFO:teuthology.orchestra.run.smithi086.stdout:6 2023-10-16T08:38:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.421+0000 7ffbd80eb640 1 -- 172.21.15.86:0/2664341499 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffbd0066190 msgr2=0x7ffbd0156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.421+0000 7ffbd80eb640 1 --2- 172.21.15.86:0/2664341499 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffbd0066190 0x7ffbd0156280 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7ffbcc00c500 tx=0x7ffbcc00c9c0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.421+0000 7ffbd80eb640 1 -- 172.21.15.86:0/2664341499 shutdown_connections 2023-10-16T08:38:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.421+0000 7ffbd80eb640 1 --2- 172.21.15.86:0/2664341499 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffbd01614c0 0x7ffbd01548a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.425 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.421+0000 7ffbd80eb640 1 --2- 172.21.15.86:0/2664341499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbd01602a0 0x7ffbd0157d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.421+0000 7ffbd80eb640 1 --2- 172.21.15.86:0/2664341499 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffbd0066190 0x7ffbd0156280 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.421+0000 7ffbd80eb640 1 -- 172.21.15.86:0/2664341499 >> 172.21.15.86:0/2664341499 conn(0x7ffbd00cc2d0 msgr2=0x7ffbd00adda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.421+0000 7ffbd80eb640 1 -- 172.21.15.86:0/2664341499 shutdown_connections 2023-10-16T08:38:22.426 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.421+0000 7ffbd80eb640 1 -- 172.21.15.86:0/2664341499 wait complete. 2023-10-16T08:38:22.440 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --cluster ceph osd new 0d7d34a3-a570-49fd-b542-25fd99bdf35b 7 2023-10-16T08:38:22.473 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:22.469+0000 7f5901f19180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-16T08:38:22.476 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:22.473+0000 7f284cdce180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T08:38:22.544 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.541+0000 7f196f437640 1 Processor -- start 2023-10-16T08:38:22.545 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.541+0000 7f196f437640 1 -- start start 2023-10-16T08:38:22.547 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f196f437640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f196806dd70 0x7f19680ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.547 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f196f437640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19680cffd0 con 0x7f19680ceb00 2023-10-16T08:38:22.547 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f196f437640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19680cf310 con 0x7f196815ad80 2023-10-16T08:38:22.548 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f196f437640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19680cf5f0 con 0x7f196806dd70 2023-10-16T08:38:22.548 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f196d1d3640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f196815ad80 0x7f196806d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.86:44438/0 (socket says 172.21.15.86:44438) 2023-10-16T08:38:22.548 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f196d1d3640 1 -- 172.21.15.86:0/373632879 learned_addr learned my addr 172.21.15.86:0/373632879 (peer_addr_for_me v1:172.21.15.86:0/0) 2023-10-16T08:38:22.548 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f196c9d2640 1 --2- 172.21.15.86:0/373632879 >> v2:172.21.15.40:3300/0 conn(0x7f196806dd70 0x7f19680ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.548 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f195bfff640 1 -- 172.21.15.86:0/373632879 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 594765675 0 0) 0x7f19680cf310 con 0x7f196815ad80 2023-10-16T08:38:22.549 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f195bfff640 1 -- 172.21.15.86:0/373632879 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1954003530 con 0x7f196815ad80 2023-10-16T08:38:22.549 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f195bfff640 1 -- 172.21.15.86:0/373632879 <== mon.1 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3014637634 0 0) 0x7f1954003530 con 0x7f196815ad80 2023-10-16T08:38:22.549 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f195bfff640 1 -- 172.21.15.86:0/373632879 >> v2:172.21.15.40:3300/0 conn(0x7f196806dd70 msgr2=0x7f19680ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:22.549 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f195bfff640 1 --2- 172.21.15.86:0/373632879 >> v2:172.21.15.40:3300/0 conn(0x7f196806dd70 0x7f19680ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.549 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f195bfff640 1 -- 172.21.15.86:0/373632879 >> v1:172.21.15.40:6789/0 conn(0x7f19680ceb00 legacy=0x7f19680d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:22.549 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f195bfff640 1 -- 172.21.15.86:0/373632879 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19680cf8d0 con 0x7f196815ad80 2023-10-16T08:38:22.550 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f195bfff640 1 -- 172.21.15.86:0/373632879 <== mon.1 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f195c001680 con 0x7f196815ad80 2023-10-16T08:38:22.550 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f195bfff640 1 -- 172.21.15.86:0/373632879 >> v1:172.21.15.86:6789/0 conn(0x7f196815ad80 legacy=0x7f196806d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:22.550 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f195bfff640 1 --2- 172.21.15.86:0/373632879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19540039f0 0x7f1954003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.550 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f195bfff640 1 --2- 172.21.15.86:0/373632879 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f19540047d0 0x7f1954004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.550 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f195bfff640 1 --2- 172.21.15.86:0/373632879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1954005190 0x7f195400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f195bfff640 1 -- 172.21.15.86:0/373632879 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19680cf5f0 con 0x7f1954005190 2023-10-16T08:38:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f195bfff640 1 -- 172.21.15.86:0/373632879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195400c0c0 con 0x7f19540039f0 2023-10-16T08:38:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f195bfff640 1 -- 172.21.15.86:0/373632879 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195400c280 con 0x7f19540047d0 2023-10-16T08:38:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f196d1d3640 1 --2- 172.21.15.86:0/373632879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19540039f0 0x7f1954003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.551 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f196c9d2640 1 --2- 172.21.15.86:0/373632879 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f19540047d0 0x7f1954004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.552 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f196d9d4640 1 --2- 172.21.15.86:0/373632879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1954005190 0x7f195400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.552 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f196d1d3640 1 -- 172.21.15.86:0/373632879 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f19540047d0 msgr2=0x7f1954004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:22.552 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f196d1d3640 1 --2- 172.21.15.86:0/373632879 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f19540047d0 0x7f1954004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.552 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f196d1d3640 1 -- 172.21.15.86:0/373632879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1954005190 msgr2=0x7f195400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:22.552 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f196d1d3640 1 --2- 172.21.15.86:0/373632879 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1954005190 0x7f195400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.553 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f196d9d4640 1 -- 172.21.15.86:0/373632879 reap_dead start 2023-10-16T08:38:22.553 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f196d1d3640 1 -- 172.21.15.86:0/373632879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1954003530 con 0x7f19540039f0 2023-10-16T08:38:22.556 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f196d1d3640 1 --2- 172.21.15.86:0/373632879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19540039f0 0x7f1954003e10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c240 tx=0x7f195c01cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0ac9a4e64de8f33 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:22.556 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f195bfff640 1 -- 172.21.15.86:0/373632879 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c0a5b40 con 0x7f19540039f0 2023-10-16T08:38:22.557 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f195bfff640 1 -- 172.21.15.86:0/373632879 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c0a5ca0 con 0x7f19540039f0 2023-10-16T08:38:22.557 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.545+0000 7f195bfff640 1 -- 172.21.15.86:0/373632879 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c005b20 con 0x7f19540039f0 2023-10-16T08:38:22.557 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196f437640 1 -- 172.21.15.86:0/373632879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19540039f0 msgr2=0x7f1954003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:22.557 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196f437640 1 --2- 172.21.15.86:0/373632879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19540039f0 0x7f1954003e10 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c240 tx=0x7f195c01cc40 comp rx=0 tx=0).stop 2023-10-16T08:38:22.557 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196f437640 1 -- 172.21.15.86:0/373632879 shutdown_connections 2023-10-16T08:38:22.558 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196f437640 1 --2- 172.21.15.86:0/373632879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19540039f0 0x7f1954003e10 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.558 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196f437640 1 -- 172.21.15.86:0/373632879 >> 172.21.15.86:0/373632879 conn(0x7f1968071580 msgr2=0x7f1968071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:22.558 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196f437640 1 -- 172.21.15.86:0/373632879 shutdown_connections 2023-10-16T08:38:22.558 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196f437640 1 -- 172.21.15.86:0/373632879 wait complete. 2023-10-16T08:38:22.558 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196f437640 1 Processor -- start 2023-10-16T08:38:22.559 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196f437640 1 -- start start 2023-10-16T08:38:22.559 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196f437640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f19540039f0 0x7f19681539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.559 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196f437640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19680b4bd0 0x7f1968153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.559 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196f437640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19680b3570 0x7f196806a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.559 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196f437640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1968153850 con 0x7f19680b3570 2023-10-16T08:38:22.560 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196f437640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19680b3b30 con 0x7f19680b4bd0 2023-10-16T08:38:22.560 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196f437640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19680b3e10 con 0x7f19540039f0 2023-10-16T08:38:22.560 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196d1d3640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f19540039f0 0x7f19681539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.560 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196d1d3640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f19540039f0 0x7f19681539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.86:48770/0 (socket says 172.21.15.86:48770) 2023-10-16T08:38:22.560 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196d1d3640 1 -- 172.21.15.86:0/828544155 learned_addr learned my addr 172.21.15.86:0/828544155 (peer_addr_for_me v2:172.21.15.86:0/0) 2023-10-16T08:38:22.560 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196c9d2640 1 --2- 172.21.15.86:0/828544155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19680b4bd0 0x7f1968153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.561 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196d9d4640 1 --2- 172.21.15.86:0/828544155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19680b3570 0x7f196806a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.561 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196d1d3640 1 -- 172.21.15.86:0/828544155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19680b4bd0 msgr2=0x7f1968153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:22.561 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196d1d3640 1 --2- 172.21.15.86:0/828544155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19680b4bd0 0x7f1968153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.561 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196d1d3640 1 -- 172.21.15.86:0/828544155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19680b3570 msgr2=0x7f196806a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:22.561 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196d1d3640 1 --2- 172.21.15.86:0/828544155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19680b3570 0x7f196806a550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.562 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196d1d3640 1 -- 172.21.15.86:0/828544155 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19680b40f0 con 0x7f19540039f0 2023-10-16T08:38:22.562 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196c9d2640 1 --2- 172.21.15.86:0/828544155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19680b4bd0 0x7f1968153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:38:22.562 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196d9d4640 1 --2- 172.21.15.86:0/828544155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19680b3570 0x7f196806a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:38:22.562 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196d1d3640 1 --2- 172.21.15.86:0/828544155 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f19540039f0 0x7f19681539f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c240 tx=0x7f1954005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:22.562 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f1959ffb640 1 -- 172.21.15.86:0/828544155 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c0b3070 con 0x7f19540039f0 2023-10-16T08:38:22.563 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f1959ffb640 1 -- 172.21.15.86:0/828544155 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f195c004e50 con 0x7f19540039f0 2023-10-16T08:38:22.563 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f1959ffb640 1 -- 172.21.15.86:0/828544155 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f195c001680 con 0x7f19540039f0 2023-10-16T08:38:22.563 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196f437640 1 -- 172.21.15.86:0/828544155 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19680b43d0 con 0x7f19540039f0 2023-10-16T08:38:22.563 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f196f437640 1 -- 172.21.15.86:0/828544155 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1968154170 con 0x7f19540039f0 2023-10-16T08:38:22.564 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f1959ffb640 1 -- 172.21.15.86:0/828544155 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f195c007140 con 0x7f19540039f0 2023-10-16T08:38:22.564 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.549+0000 7f1959ffb640 1 -- 172.21.15.86:0/828544155 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7f195c0aebb0 con 0x7f19540039f0 2023-10-16T08:38:22.564 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.553+0000 7f196f437640 1 -- 172.21.15.86:0/828544155 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196804e9c0 con 0x7f19540039f0 2023-10-16T08:38:22.564 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.553+0000 7f1959ffb640 1 -- 172.21.15.86:0/828544155 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f195c0be060 con 0x7f19540039f0 2023-10-16T08:38:22.618 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.617+0000 7f196f437640 1 -- 172.21.15.86:0/828544155 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd new", "uuid": "0d7d34a3-a570-49fd-b542-25fd99bdf35b", "id": 7} v 0) v1 -- 0x7f19680b01a0 con 0x7f19540039f0 2023-10-16T08:38:22.619 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:22.617+0000 7f5901f19180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-16T08:38:22.630 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.629+0000 7f1959ffb640 1 -- 172.21.15.86:0/828544155 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "0d7d34a3-a570-49fd-b542-25fd99bdf35b", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f195c007cf0 con 0x7f19540039f0 2023-10-16T08:38:22.631 INFO:teuthology.orchestra.run.smithi086.stdout:7 2023-10-16T08:38:22.632 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.629+0000 7f196f437640 1 -- 172.21.15.86:0/828544155 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f19540039f0 msgr2=0x7f19681539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:22.632 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.629+0000 7f196f437640 1 --2- 172.21.15.86:0/828544155 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f19540039f0 0x7f19681539f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f195c00c240 tx=0x7f1954005800 comp rx=0 tx=0).stop 2023-10-16T08:38:22.632 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.629+0000 7f196f437640 1 -- 172.21.15.86:0/828544155 shutdown_connections 2023-10-16T08:38:22.632 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.629+0000 7f196f437640 1 --2- 172.21.15.86:0/828544155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f19680b3570 0x7f196806a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.632 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.629+0000 7f196f437640 1 --2- 172.21.15.86:0/828544155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f19680b4bd0 0x7f1968153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.633 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.629+0000 7f196f437640 1 --2- 172.21.15.86:0/828544155 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f19540039f0 0x7f19681539f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.633 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.629+0000 7f196f437640 1 -- 172.21.15.86:0/828544155 >> 172.21.15.86:0/828544155 conn(0x7f1968071580 msgr2=0x7f19681585c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:22.633 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.629+0000 7f196f437640 1 -- 172.21.15.86:0/828544155 shutdown_connections 2023-10-16T08:38:22.633 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:38:22.629+0000 7f196f437640 1 -- 172.21.15.86:0/828544155 wait complete. 2023-10-16T08:38:22.646 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-16T08:38:22.647 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-16T08:38:22.649 INFO:tasks.ceph.osd.0:Started 2023-10-16T08:38:22.649 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-16T08:38:22.650 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-16T08:38:22.652 INFO:tasks.ceph.osd.1:Started 2023-10-16T08:38:22.653 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-16T08:38:22.653 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-16T08:38:22.656 INFO:tasks.ceph.osd.2:Started 2023-10-16T08:38:22.656 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-16T08:38:22.657 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-16T08:38:22.659 INFO:tasks.ceph.osd.3:Started 2023-10-16T08:38:22.660 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-16T08:38:22.660 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-16T08:38:22.684 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:22.681+0000 7f5901f19180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-16T08:38:22.690 INFO:tasks.ceph.osd.4:Started 2023-10-16T08:38:22.691 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-16T08:38:22.691 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-16T08:38:22.693 INFO:tasks.ceph.osd.5:Started 2023-10-16T08:38:22.694 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-16T08:38:22.694 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-16T08:38:22.696 INFO:tasks.ceph.osd.6:Started 2023-10-16T08:38:22.697 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-16T08:38:22.697 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-16T08:38:22.699 INFO:tasks.ceph.osd.7:Started 2023-10-16T08:38:22.700 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:38:22.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.821+0000 7f86cbe45640 1 Processor -- start 2023-10-16T08:38:22.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.821+0000 7f86cbe45640 1 -- start start 2023-10-16T08:38:22.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.821+0000 7f86cbe45640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86c40cbf50 0x7f86c4065a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.821+0000 7f86cbe45640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f86c406e0b0 0x7f86c4065fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.821+0000 7f86cbe45640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86c415ac10 con 0x7f86c406dcd0 2023-10-16T08:38:22.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.821+0000 7f86cbe45640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86c40cf330 con 0x7f86c40cbf50 2023-10-16T08:38:22.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.821+0000 7f86cbe45640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c40cf600 con 0x7f86c406e0b0 2023-10-16T08:38:22.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.821+0000 7f86c93e0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86c40cbf50 0x7f86c4065a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.821+0000 7f86c93e0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f86c40cbf50 0x7f86c4065a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50676/0 (socket says 172.21.15.40:50676) 2023-10-16T08:38:22.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.821+0000 7f86c93e0640 1 -- 172.21.15.40:0/479830571 learned_addr learned my addr 172.21.15.40:0/479830571 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:22.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.821+0000 7f86ca3e2640 1 --2- 172.21.15.40:0/479830571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f86c406e0b0 0x7f86c4065fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.821+0000 7f86c93e0640 1 -- 172.21.15.40:0/479830571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f86c406e0b0 msgr2=0x7f86c4065fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:22.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.821+0000 7f86c93e0640 1 --2- 172.21.15.40:0/479830571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f86c406e0b0 0x7f86c4065fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.821+0000 7f86c93e0640 1 -- 172.21.15.40:0/479830571 >> v1:172.21.15.86:6789/0 conn(0x7f86c406dcd0 legacy=0x7f86c40cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:22.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.821+0000 7f86c93e0640 1 -- 172.21.15.40:0/479830571 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86c4070a20 con 0x7f86c40cbf50 2023-10-16T08:38:22.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.825+0000 7f86c93e0640 1 --2- 172.21.15.40:0/479830571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c40cbf50 0x7f86c4065a80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009870 tx=0x7f86b401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2552abda1801764 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:22.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.825+0000 7f86c8bdf640 1 -- 172.21.15.40:0/479830571 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b40aa670 con 0x7f86c40cbf50 2023-10-16T08:38:22.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.825+0000 7f86c8bdf640 1 -- 172.21.15.40:0/479830571 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86b40aa7d0 con 0x7f86c40cbf50 2023-10-16T08:38:22.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.825+0000 7f86c8bdf640 1 -- 172.21.15.40:0/479830571 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86b40aab60 con 0x7f86c40cbf50 2023-10-16T08:38:22.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.825+0000 7f86cbe45640 1 -- 172.21.15.40:0/479830571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c40cbf50 msgr2=0x7f86c4065a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:22.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.825+0000 7f86cbe45640 1 --2- 172.21.15.40:0/479830571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c40cbf50 0x7f86c4065a80 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f86b4009870 tx=0x7f86b401c510 comp rx=0 tx=0).stop 2023-10-16T08:38:22.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.825+0000 7f86cbe45640 1 -- 172.21.15.40:0/479830571 shutdown_connections 2023-10-16T08:38:22.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.825+0000 7f86cbe45640 1 --2- 172.21.15.40:0/479830571 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f86c406e0b0 0x7f86c4065fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.825+0000 7f86cbe45640 1 --2- 172.21.15.40:0/479830571 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c40cbf50 0x7f86c4065a80 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.825+0000 7f86cbe45640 1 -- 172.21.15.40:0/479830571 >> 172.21.15.40:0/479830571 conn(0x7f86c40cc410 msgr2=0x7f86c40cc820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:22.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.825+0000 7f86cbe45640 1 -- 172.21.15.40:0/479830571 shutdown_connections 2023-10-16T08:38:22.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.825+0000 7f86cbe45640 1 -- 172.21.15.40:0/479830571 wait complete. 2023-10-16T08:38:22.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.825+0000 7f86cbe45640 1 Processor -- start 2023-10-16T08:38:22.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.825+0000 7f86cbe45640 1 -- start start 2023-10-16T08:38:22.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86cbe45640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86c406dcd0 0x7f86c414e120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86cbe45640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f86c406e0b0 0x7f86c4159c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86cbe45640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c40cbf50 0x7f86c414e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:22.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86cbe45640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c4158b70 con 0x7f86c40cbf50 2023-10-16T08:38:22.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86cbe45640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86c4158e50 con 0x7f86c406dcd0 2023-10-16T08:38:22.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86cbe45640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86c4159130 con 0x7f86c406e0b0 2023-10-16T08:38:22.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86ca3e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c40cbf50 0x7f86c414e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86ca3e2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c40cbf50 0x7f86c414e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50680/0 (socket says 172.21.15.40:50680) 2023-10-16T08:38:22.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86ca3e2640 1 -- 172.21.15.40:0/2587302220 learned_addr learned my addr 172.21.15.40:0/2587302220 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:22.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86c93e0640 1 --2- 172.21.15.40:0/2587302220 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f86c406e0b0 0x7f86c4159c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86c9be1640 1 --2- 172.21.15.40:0/2587302220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86c406dcd0 0x7f86c414e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:22.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86ca3e2640 1 -- 172.21.15.40:0/2587302220 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f86c406e0b0 msgr2=0x7f86c4159c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:22.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86ca3e2640 1 --2- 172.21.15.40:0/2587302220 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f86c406e0b0 0x7f86c4159c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86ca3e2640 1 -- 172.21.15.40:0/2587302220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86c406dcd0 msgr2=0x7f86c414e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:22.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86ca3e2640 1 --2- 172.21.15.40:0/2587302220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86c406dcd0 0x7f86c414e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86ca3e2640 1 -- 172.21.15.40:0/2587302220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86b40094d0 con 0x7f86c40cbf50 2023-10-16T08:38:22.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86ca3e2640 1 --2- 172.21.15.40:0/2587302220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c40cbf50 0x7f86c414e930 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00efc0 tx=0x7f86bc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:22.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86ba7fc640 1 -- 172.21.15.40:0/2587302220 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc009ca0 con 0x7f86c40cbf50 2023-10-16T08:38:22.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86ba7fc640 1 -- 172.21.15.40:0/2587302220 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86bc00f040 con 0x7f86c40cbf50 2023-10-16T08:38:22.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86ba7fc640 1 -- 172.21.15.40:0/2587302220 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86bc0146c0 con 0x7f86c40cbf50 2023-10-16T08:38:22.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86cbe45640 1 -- 172.21.15.40:0/2587302220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86c4159410 con 0x7f86c40cbf50 2023-10-16T08:38:22.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86cbe45640 1 -- 172.21.15.40:0/2587302220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86c414e660 con 0x7f86c40cbf50 2023-10-16T08:38:22.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86ba7fc640 1 -- 172.21.15.40:0/2587302220 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f86bc014b20 con 0x7f86c40cbf50 2023-10-16T08:38:22.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86ba7fc640 1 -- 172.21.15.40:0/2587302220 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f86bc01dcc0 con 0x7f86c40cbf50 2023-10-16T08:38:22.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.829+0000 7f86cbe45640 1 -- 172.21.15.40:0/2587302220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86c404e9c0 con 0x7f86c40cbf50 2023-10-16T08:38:22.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.845+0000 7f86ba7fc640 1 -- 172.21.15.40:0/2587302220 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f86bc03bb40 con 0x7f86c40cbf50 2023-10-16T08:38:22.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.901+0000 7f86cbe45640 1 -- 172.21.15.40:0/2587302220 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86c41598c0 con 0x7f86c40cbf50 2023-10-16T08:38:22.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.901+0000 7f86ba7fc640 1 -- 172.21.15.40:0/2587302220 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f86bc02a070 con 0x7f86c40cbf50 2023-10-16T08:38:22.905 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:22.905 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":10,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:22.620481+0000","last_up_change":"0.000000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:38:22.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.905+0000 7f86cbe45640 1 -- 172.21.15.40:0/2587302220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c40cbf50 msgr2=0x7f86c414e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:22.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.905+0000 7f86cbe45640 1 --2- 172.21.15.40:0/2587302220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c40cbf50 0x7f86c414e930 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f86bc00efc0 tx=0x7f86bc00c490 comp rx=0 tx=0).stop 2023-10-16T08:38:22.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.905+0000 7f86cbe45640 1 -- 172.21.15.40:0/2587302220 shutdown_connections 2023-10-16T08:38:22.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.905+0000 7f86cbe45640 1 --2- 172.21.15.40:0/2587302220 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f86c40cbf50 0x7f86c414e930 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.905+0000 7f86cbe45640 1 --2- 172.21.15.40:0/2587302220 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f86c406e0b0 0x7f86c4159c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.905+0000 7f86cbe45640 1 --2- 172.21.15.40:0/2587302220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f86c406dcd0 0x7f86c414e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:22.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.905+0000 7f86cbe45640 1 -- 172.21.15.40:0/2587302220 >> 172.21.15.40:0/2587302220 conn(0x7f86c40cc410 msgr2=0x7f86c415d220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:22.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.905+0000 7f86cbe45640 1 -- 172.21.15.40:0/2587302220 shutdown_connections 2023-10-16T08:38:22.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:22.905+0000 7f86cbe45640 1 -- 172.21.15.40:0/2587302220 wait complete. 2023-10-16T08:38:22.921 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-16T08:38:22.921 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-16T08:38:23.096 INFO:tasks.ceph.mgr.x.smithi086.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-10-16T08:38:23.097 INFO:tasks.ceph.mgr.x.smithi086.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-10-16T08:38:23.097 INFO:tasks.ceph.mgr.x.smithi086.stderr: from numpy import show_config as show_numpy_config 2023-10-16T08:38:23.136 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:23.133+0000 7f284cdce180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T08:38:23.271 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:23.269+0000 7f284cdce180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T08:38:23.283 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-10-16T08:38:23.281+0000 7f290b0d2640 -1 Falling back to public interface 2023-10-16T08:38:23.296 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-10-16T08:38:23.293+0000 7fa07d80b640 -1 Falling back to public interface 2023-10-16T08:38:23.298 INFO:tasks.ceph.osd.7.smithi086.stderr:2023-10-16T08:38:23.297+0000 7f30bdd6b640 -1 Falling back to public interface 2023-10-16T08:38:23.302 INFO:tasks.ceph.osd.4.smithi086.stderr:2023-10-16T08:38:23.301+0000 7f39e3495640 -1 Falling back to public interface 2023-10-16T08:38:23.302 INFO:tasks.ceph.osd.6.smithi086.stderr:2023-10-16T08:38:23.301+0000 7f1391b13640 -1 Falling back to public interface 2023-10-16T08:38:23.310 INFO:tasks.ceph.osd.5.smithi086.stderr:2023-10-16T08:38:23.309+0000 7faee976a640 -1 Falling back to public interface 2023-10-16T08:38:23.315 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-10-16T08:38:23.313+0000 7f9594e5e640 -1 Falling back to public interface 2023-10-16T08:38:23.315 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-10-16T08:38:23.313+0000 7f7f3f70a640 -1 Falling back to public interface 2023-10-16T08:38:23.444 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:23.441+0000 7f284cdce180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T08:38:23.507 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:23.505+0000 7f284cdce180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T08:38:23.629 INFO:tasks.ceph.mgr.y.smithi040.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-10-16T08:38:23.630 INFO:tasks.ceph.mgr.y.smithi040.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-10-16T08:38:23.630 INFO:tasks.ceph.mgr.y.smithi040.stderr: from numpy import show_config as show_numpy_config 2023-10-16T08:38:23.630 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:23.629+0000 7f284cdce180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T08:38:23.688 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:23.685+0000 7f5901f19180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-16T08:38:23.726 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-16T08:38:23.828 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:23.825+0000 7f5901f19180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-16T08:38:23.846 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.845+0000 7f4261722640 1 Processor -- start 2023-10-16T08:38:23.846 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.845+0000 7f4261722640 1 -- start start 2023-10-16T08:38:23.848 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.845+0000 7f4261722640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f425c0cbf50 0x7f425c065a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:23.848 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.845+0000 7f4261722640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425c15ac10 con 0x7f425c06dcd0 2023-10-16T08:38:23.848 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.845+0000 7f4261722640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f425c0cf330 con 0x7f425c06e0b0 2023-10-16T08:38:23.848 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.845+0000 7f4261722640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f425c0cf600 con 0x7f425c0cbf50 2023-10-16T08:38:23.848 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.845+0000 7f425a7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f425c0cbf50 0x7f425c065a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:23.849 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.845+0000 7f425affd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f425c06dcd0 0x7f425c0cec20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60688/0 (socket says 172.21.15.40:60688) 2023-10-16T08:38:23.849 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.845+0000 7f425affd640 1 -- 172.21.15.40:0/1608561940 learned_addr learned my addr 172.21.15.40:0/1608561940 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:23.849 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.845+0000 7f425a7fc640 1 -- 172.21.15.40:0/1608561940 >> v1:172.21.15.86:6789/0 conn(0x7f425c06e0b0 legacy=0x7f425c160880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:23.850 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.845+0000 7f425a7fc640 1 -- 172.21.15.40:0/1608561940 >> v1:172.21.15.40:6789/0 conn(0x7f425c06dcd0 legacy=0x7f425c0cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:23.850 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.845+0000 7f425a7fc640 1 -- 172.21.15.40:0/1608561940 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f425c070a20 con 0x7f425c0cbf50 2023-10-16T08:38:23.850 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.845+0000 7f4259ffb640 1 -- 172.21.15.40:0/1608561940 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479036305 0 0) 0x7f425c15ac10 con 0x7f425c06dcd0 2023-10-16T08:38:23.851 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.845+0000 7f425a7fc640 1 --2- 172.21.15.40:0/1608561940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425c0cbf50 0x7f425c065a80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f424c009d90 tx=0x7f424c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d76f3e1d32f47a80 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:23.851 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f4259ffb640 1 -- 172.21.15.40:0/1608561940 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c0a58b0 con 0x7f425c0cbf50 2023-10-16T08:38:23.852 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f4259ffb640 1 -- 172.21.15.40:0/1608561940 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f424c0a5a10 con 0x7f425c0cbf50 2023-10-16T08:38:23.852 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f4259ffb640 1 -- 172.21.15.40:0/1608561940 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f424c0a5da0 con 0x7f425c0cbf50 2023-10-16T08:38:23.852 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f4261722640 1 -- 172.21.15.40:0/1608561940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425c0cbf50 msgr2=0x7f425c065a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:23.852 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f4261722640 1 --2- 172.21.15.40:0/1608561940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425c0cbf50 0x7f425c065a80 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f424c009d90 tx=0x7f424c01c8e0 comp rx=0 tx=0).stop 2023-10-16T08:38:23.852 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f4261722640 1 -- 172.21.15.40:0/1608561940 shutdown_connections 2023-10-16T08:38:23.853 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f4261722640 1 --2- 172.21.15.40:0/1608561940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425c0cbf50 0x7f425c065a80 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:23.853 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f4261722640 1 -- 172.21.15.40:0/1608561940 >> 172.21.15.40:0/1608561940 conn(0x7f425c0cc410 msgr2=0x7f425c0cc820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:23.853 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f4261722640 1 -- 172.21.15.40:0/1608561940 shutdown_connections 2023-10-16T08:38:23.853 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f4261722640 1 -- 172.21.15.40:0/1608561940 wait complete. 2023-10-16T08:38:23.856 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f4261722640 1 Processor -- start 2023-10-16T08:38:23.856 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f4261722640 1 -- start start 2023-10-16T08:38:23.856 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f4261722640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f425c06dcd0 0x7f425c0b1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:23.856 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f4261722640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f425c06e0b0 0x7f425c0adb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:23.857 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f4261722640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425c0cbf50 0x7f425c152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:23.857 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f4261722640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c0b07f0 con 0x7f425c0cbf50 2023-10-16T08:38:23.857 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f4261722640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f425c151c20 con 0x7f425c06e0b0 2023-10-16T08:38:23.857 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f4261722640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f425c150b80 con 0x7f425c06dcd0 2023-10-16T08:38:23.857 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f425b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425c0cbf50 0x7f425c152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:23.858 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f425b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425c0cbf50 0x7f425c152810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50690/0 (socket says 172.21.15.40:50690) 2023-10-16T08:38:23.858 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f425b7fe640 1 -- 172.21.15.40:0/833899007 learned_addr learned my addr 172.21.15.40:0/833899007 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:23.858 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f425a7fc640 1 --2- 172.21.15.40:0/833899007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f425c06e0b0 0x7f425c0adb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:23.858 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f425affd640 1 --2- 172.21.15.40:0/833899007 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f425c06dcd0 0x7f425c0b1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:23.858 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f425b7fe640 1 -- 172.21.15.40:0/833899007 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f425c06dcd0 msgr2=0x7f425c0b1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:23.859 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f425b7fe640 1 --2- 172.21.15.40:0/833899007 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f425c06dcd0 0x7f425c0b1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:23.859 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f425b7fe640 1 -- 172.21.15.40:0/833899007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f425c06e0b0 msgr2=0x7f425c0adb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:23.859 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f425b7fe640 1 --2- 172.21.15.40:0/833899007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f425c06e0b0 0x7f425c0adb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:23.859 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f425b7fe640 1 -- 172.21.15.40:0/833899007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f424c009a70 con 0x7f425c0cbf50 2023-10-16T08:38:23.859 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f425affd640 1 --2- 172.21.15.40:0/833899007 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f425c06dcd0 0x7f425c0b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:38:23.860 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f425b7fe640 1 --2- 172.21.15.40:0/833899007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425c0cbf50 0x7f425c152810 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f425400c390 tx=0x7f425400c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:23.860 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f42437fe640 1 -- 172.21.15.40:0/833899007 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254011790 con 0x7f425c0cbf50 2023-10-16T08:38:23.860 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f42437fe640 1 -- 172.21.15.40:0/833899007 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42540118f0 con 0x7f425c0cbf50 2023-10-16T08:38:23.860 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f4261722640 1 -- 172.21.15.40:0/833899007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f425c150e60 con 0x7f425c0cbf50 2023-10-16T08:38:23.860 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.849+0000 7f42437fe640 1 -- 172.21.15.40:0/833899007 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4254011ca0 con 0x7f425c0cbf50 2023-10-16T08:38:23.860 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.853+0000 7f4261722640 1 -- 172.21.15.40:0/833899007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f425c151420 con 0x7f425c0cbf50 2023-10-16T08:38:23.861 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.853+0000 7f42437fe640 1 -- 172.21.15.40:0/833899007 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f425401c760 con 0x7f425c0cbf50 2023-10-16T08:38:23.861 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.853+0000 7f42437fe640 1 -- 172.21.15.40:0/833899007 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f4254012dc0 con 0x7f425c0cbf50 2023-10-16T08:38:23.861 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.853+0000 7f4261722640 1 -- 172.21.15.40:0/833899007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f425c04e9c0 con 0x7f425c0cbf50 2023-10-16T08:38:23.862 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.853+0000 7f42437fe640 1 -- 172.21.15.40:0/833899007 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72483 (secure 0 0 0) 0x7f4254011e00 con 0x7f425c0cbf50 2023-10-16T08:38:23.914 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.913+0000 7f4261722640 1 -- 172.21.15.40:0/833899007 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f425c071290 con 0x7f425c0cbf50 2023-10-16T08:38:23.915 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.913+0000 7f42437fe640 1 -- 172.21.15.40:0/833899007 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f42540126f0 con 0x7f425c0cbf50 2023-10-16T08:38:23.916 INFO:teuthology.misc.health.smithi040.stdout: 2023-10-16T08:38:23.916 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":10,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:22.620481+0000","last_up_change":"0.000000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:38:23.917 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.913+0000 7f4261722640 1 -- 172.21.15.40:0/833899007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425c0cbf50 msgr2=0x7f425c152810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:23.917 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.913+0000 7f4261722640 1 --2- 172.21.15.40:0/833899007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425c0cbf50 0x7f425c152810 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f425400c390 tx=0x7f425400c850 comp rx=0 tx=0).stop 2023-10-16T08:38:23.917 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.913+0000 7f4261722640 1 -- 172.21.15.40:0/833899007 shutdown_connections 2023-10-16T08:38:23.917 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.913+0000 7f4261722640 1 --2- 172.21.15.40:0/833899007 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f425c0cbf50 0x7f425c152810 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:23.917 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.913+0000 7f4261722640 1 --2- 172.21.15.40:0/833899007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f425c06e0b0 0x7f425c0adb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:23.918 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.913+0000 7f4261722640 1 --2- 172.21.15.40:0/833899007 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f425c06dcd0 0x7f425c0b1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:23.918 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.913+0000 7f4261722640 1 -- 172.21.15.40:0/833899007 >> 172.21.15.40:0/833899007 conn(0x7f425c0cc410 msgr2=0x7f425c0c9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:23.918 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.913+0000 7f4261722640 1 -- 172.21.15.40:0/833899007 shutdown_connections 2023-10-16T08:38:23.918 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:23.913+0000 7f4261722640 1 -- 172.21.15.40:0/833899007 wait complete. 2023-10-16T08:38:23.930 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-16T08:38:24.086 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:24.085+0000 7f5901f19180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-16T08:38:24.153 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:24.149+0000 7f5901f19180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-16T08:38:24.238 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:24.237+0000 7f284cdce180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T08:38:24.283 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:24.281+0000 7f5901f19180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-16T08:38:24.313 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:24.309+0000 7f284cdce180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T08:38:24.481 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:24.481+0000 7f284cdce180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T08:38:24.555 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:24.553+0000 7f284cdce180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T08:38:24.839 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:24.837+0000 7f284cdce180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T08:38:24.900 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:24.897+0000 7f284cdce180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T08:38:24.913 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:24.909+0000 7f5901f19180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-16T08:38:24.999 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:24.997+0000 7f5901f19180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-16T08:38:25.179 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:25.177+0000 7f5901f19180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-16T08:38:25.255 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:25.253+0000 7f5901f19180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-16T08:38:25.559 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:25.557+0000 7f5901f19180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-16T08:38:25.622 INFO:tasks.ceph.mgr.y.smithi040.stderr:2023-10-16T08:38:25.621+0000 7f5901f19180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-16T08:38:25.948 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:25.945+0000 7f27f4542640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-16T08:38:25.948 INFO:tasks.ceph.mgr.x.smithi086.stderr:2023-10-16T08:38:25.945+0000 7f27f4542640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-16T08:38:27.090 INFO:tasks.ceph.osd.4.smithi086.stderr:2023-10-16T08:38:27.089+0000 7f39e3495640 -1 osd.4 0 log_to_monitors true 2023-10-16T08:38:27.167 INFO:tasks.ceph.osd.5.smithi086.stderr:2023-10-16T08:38:27.165+0000 7faee976a640 -1 osd.5 0 log_to_monitors true 2023-10-16T08:38:27.172 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-10-16T08:38:27.168+0000 7f290b0d2640 -1 osd.2 0 log_to_monitors true 2023-10-16T08:38:27.189 INFO:tasks.ceph.osd.6.smithi086.stderr:2023-10-16T08:38:27.189+0000 7f1391b13640 -1 osd.6 0 log_to_monitors true 2023-10-16T08:38:27.216 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-10-16T08:38:27.212+0000 7fa07d80b640 -1 osd.3 0 log_to_monitors true 2023-10-16T08:38:27.273 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-10-16T08:38:27.268+0000 7f9594e5e640 -1 osd.0 0 log_to_monitors true 2023-10-16T08:38:27.273 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-10-16T08:38:27.268+0000 7f7f3f70a640 -1 osd.1 0 log_to_monitors true 2023-10-16T08:38:27.286 INFO:tasks.ceph.osd.7.smithi086.stderr:2023-10-16T08:38:27.285+0000 7f30bdd6b640 -1 osd.7 0 log_to_monitors true 2023-10-16T08:38:30.737 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-16T08:38:30.930 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.924+0000 7fb2204d8640 1 Processor -- start 2023-10-16T08:38:30.930 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.924+0000 7fb2204d8640 1 -- start start 2023-10-16T08:38:30.930 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.924+0000 7fb2204d8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb218156a20 0x7fb218159e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:30.930 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.924+0000 7fb2204d8640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2180adb80 0x7fb2180cc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:30.931 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.924+0000 7fb2204d8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2180cb370 0x7fb2180cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:30.931 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.924+0000 7fb2204d8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2180cb0a0 con 0x7fb218156a20 2023-10-16T08:38:30.933 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.924+0000 7fb2204d8640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2180cbd40 con 0x7fb2180adb80 2023-10-16T08:38:30.933 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.924+0000 7fb2204d8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb218156e00 con 0x7fb2180cb370 2023-10-16T08:38:30.933 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.924+0000 7fb21e274640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb218156a20 0x7fb218159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:30.933 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.924+0000 7fb21e274640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb218156a20 0x7fb218159e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50338/0 (socket says 172.21.15.40:50338) 2023-10-16T08:38:30.934 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.924+0000 7fb21e274640 1 -- 172.21.15.40:0/2367130673 learned_addr learned my addr 172.21.15.40:0/2367130673 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:30.934 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.924+0000 7fb21ea75640 1 --2- 172.21.15.40:0/2367130673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2180cb370 0x7fb2180cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:30.934 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.924+0000 7fb21ea75640 1 -- 172.21.15.40:0/2367130673 >> v2:172.21.15.86:3300/0 conn(0x7fb2180adb80 msgr2=0x7fb2180cc530 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-16T08:38:30.934 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.924+0000 7fb21ea75640 1 --2- 172.21.15.40:0/2367130673 >> v2:172.21.15.86:3300/0 conn(0x7fb2180adb80 0x7fb2180cc530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:30.934 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.924+0000 7fb21ea75640 1 -- 172.21.15.40:0/2367130673 >> v2:172.21.15.40:3300/0 conn(0x7fb218156a20 msgr2=0x7fb218159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:30.935 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.924+0000 7fb21ea75640 1 --2- 172.21.15.40:0/2367130673 >> v2:172.21.15.40:3300/0 conn(0x7fb218156a20 0x7fb218159e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:30.935 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.924+0000 7fb21ea75640 1 -- 172.21.15.40:0/2367130673 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2180ae0d0 con 0x7fb2180cb370 2023-10-16T08:38:30.936 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.928+0000 7fb21ea75640 1 --2- 172.21.15.40:0/2367130673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2180cb370 0x7fb2180cb800 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb218071930 tx=0x7fb2100253b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efa6f2a9347267cf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:30.936 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.928+0000 7fb21d272640 1 -- 172.21.15.40:0/2367130673 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb21000d2d0 con 0x7fb2180cb370 2023-10-16T08:38:30.936 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.928+0000 7fb21d272640 1 -- 172.21.15.40:0/2367130673 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2100b0050 con 0x7fb2180cb370 2023-10-16T08:38:30.937 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb2204d8640 1 -- 172.21.15.40:0/2367130673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2180cb370 msgr2=0x7fb2180cb800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:30.937 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb2204d8640 1 --2- 172.21.15.40:0/2367130673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2180cb370 0x7fb2180cb800 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb218071930 tx=0x7fb2100253b0 comp rx=0 tx=0).stop 2023-10-16T08:38:30.937 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb2204d8640 1 -- 172.21.15.40:0/2367130673 shutdown_connections 2023-10-16T08:38:30.937 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb2204d8640 1 --2- 172.21.15.40:0/2367130673 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2180cb370 0x7fb2180cb800 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:30.937 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb2204d8640 1 --2- 172.21.15.40:0/2367130673 >> v2:172.21.15.86:3300/0 conn(0x7fb2180adb80 0x7fb2180cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:30.938 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb2204d8640 1 --2- 172.21.15.40:0/2367130673 >> v2:172.21.15.40:3300/0 conn(0x7fb218156a20 0x7fb218159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:30.938 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb2204d8640 1 -- 172.21.15.40:0/2367130673 >> 172.21.15.40:0/2367130673 conn(0x7fb2180b0470 msgr2=0x7fb2180b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:30.938 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb2204d8640 1 -- 172.21.15.40:0/2367130673 shutdown_connections 2023-10-16T08:38:30.940 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb2204d8640 1 -- 172.21.15.40:0/2367130673 wait complete. 2023-10-16T08:38:30.941 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb2204d8640 1 Processor -- start 2023-10-16T08:38:30.941 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb2204d8640 1 -- start start 2023-10-16T08:38:30.941 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb2204d8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2180adb80 0x7fb218151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:30.941 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb2204d8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2180cb370 0x7fb218151d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:30.941 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb2204d8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb218156a20 0x7fb218152240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:30.942 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb2204d8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21814f1c0 con 0x7fb218156a20 2023-10-16T08:38:30.942 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb2204d8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb21814e160 con 0x7fb2180adb80 2023-10-16T08:38:30.942 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb2204d8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb21814e400 con 0x7fb2180cb370 2023-10-16T08:38:30.942 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb21ea75640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb218156a20 0x7fb218152240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:30.942 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb21ea75640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb218156a20 0x7fb218152240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50348/0 (socket says 172.21.15.40:50348) 2023-10-16T08:38:30.942 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb21ea75640 1 -- 172.21.15.40:0/4123483283 learned_addr learned my addr 172.21.15.40:0/4123483283 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:30.943 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.932+0000 7fb21e274640 1 --2- 172.21.15.40:0/4123483283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2180adb80 0x7fb218151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:30.943 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.936+0000 7fb21ea75640 1 -- 172.21.15.40:0/4123483283 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2180cb370 msgr2=0x7fb218151d00 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T08:38:30.943 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.936+0000 7fb21ea75640 1 --2- 172.21.15.40:0/4123483283 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2180cb370 0x7fb218151d00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:30.943 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.936+0000 7fb21ea75640 1 -- 172.21.15.40:0/4123483283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2180adb80 msgr2=0x7fb218151370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:30.943 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.936+0000 7fb21ea75640 1 --2- 172.21.15.40:0/4123483283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2180adb80 0x7fb218151370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:30.944 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.936+0000 7fb21ea75640 1 -- 172.21.15.40:0/4123483283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb210007250 con 0x7fb218156a20 2023-10-16T08:38:30.944 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.936+0000 7fb21ea75640 1 --2- 172.21.15.40:0/4123483283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb218156a20 0x7fb218152240 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb210023a60 tx=0x7fb210022150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:30.944 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.936+0000 7fb20effd640 1 -- 172.21.15.40:0/4123483283 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2100be640 con 0x7fb218156a20 2023-10-16T08:38:30.944 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.936+0000 7fb2204d8640 1 -- 172.21.15.40:0/4123483283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb21814e6e0 con 0x7fb218156a20 2023-10-16T08:38:30.944 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.936+0000 7fb2204d8640 1 -- 172.21.15.40:0/4123483283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb21814eca0 con 0x7fb218156a20 2023-10-16T08:38:30.944 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.936+0000 7fb20effd640 1 -- 172.21.15.40:0/4123483283 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2100be7a0 con 0x7fb218156a20 2023-10-16T08:38:30.945 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.936+0000 7fb20effd640 1 -- 172.21.15.40:0/4123483283 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2100beb10 con 0x7fb218156a20 2023-10-16T08:38:30.945 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.936+0000 7fb2204d8640 1 -- 172.21.15.40:0/4123483283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb200005260 con 0x7fb218156a20 2023-10-16T08:38:30.945 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.936+0000 7fb20effd640 1 -- 172.21.15.40:0/4123483283 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 4) v1 ==== 85794+0+0 (secure 0 0 0) 0x7fb21000d2e0 con 0x7fb218156a20 2023-10-16T08:38:30.946 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.936+0000 7fb20effd640 1 --2- 172.21.15.40:0/4123483283 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb1fc066440 0x7fb1fc068900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:30.946 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.936+0000 7fb20effd640 1 -- 172.21.15.40:0/4123483283 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7fb210126170 con 0x7fb218156a20 2023-10-16T08:38:30.946 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.936+0000 7fb21e274640 1 --2- 172.21.15.40:0/4123483283 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb1fc066440 0x7fb1fc068900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:30.946 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.940+0000 7fb21e274640 1 --2- 172.21.15.40:0/4123483283 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb1fc066440 0x7fb1fc068900 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb21806d520 tx=0x7fb21400d040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:30.950 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:30.948+0000 7fb20effd640 1 -- 172.21.15.40:0/4123483283 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fb2100f44f0 con 0x7fb218156a20 2023-10-16T08:38:31.171 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-10-16T08:38:31.168+0000 7f2906857640 -1 osd.2 0 waiting for initial osdmap 2023-10-16T08:38:31.200 INFO:tasks.ceph.osd.5.smithi086.stderr:2023-10-16T08:38:31.197+0000 7faee6727640 -1 osd.5 0 waiting for initial osdmap 2023-10-16T08:38:31.202 INFO:tasks.ceph.osd.2.smithi040.stderr:2023-10-16T08:38:31.200+0000 7f28fee9e640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T08:38:31.209 INFO:tasks.ceph.osd.7.smithi086.stderr:2023-10-16T08:38:31.205+0000 7f30b94f0640 -1 osd.7 0 waiting for initial osdmap 2023-10-16T08:38:31.229 INFO:tasks.ceph.osd.5.smithi086.stderr:2023-10-16T08:38:31.229+0000 7faedd536640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T08:38:31.237 INFO:tasks.ceph.osd.7.smithi086.stderr:2023-10-16T08:38:31.237+0000 7f30b1b37640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T08:38:31.253 INFO:tasks.ceph.osd.6.smithi086.stderr:2023-10-16T08:38:31.253+0000 7f138ead0640 -1 osd.6 0 waiting for initial osdmap 2023-10-16T08:38:31.274 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-10-16T08:38:31.268+0000 7f95905e3640 -1 osd.0 0 waiting for initial osdmap 2023-10-16T08:38:31.284 INFO:tasks.ceph.osd.4.smithi086.stderr:2023-10-16T08:38:31.281+0000 7f39e0452640 -1 osd.4 0 waiting for initial osdmap 2023-10-16T08:38:31.291 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:31.288+0000 7fb2204d8640 1 -- 172.21.15.40:0/4123483283 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb200005510 con 0x7fb218156a20 2023-10-16T08:38:31.292 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:31.288+0000 7fb20effd640 1 -- 172.21.15.40:0/4123483283 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fb2100f7b10 con 0x7fb218156a20 2023-10-16T08:38:31.292 INFO:teuthology.misc.health.smithi040.stdout: 2023-10-16T08:38:31.292 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":12,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:28.953819+0000","last_up_change":"0.000000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:38:31.295 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:31.292+0000 7fb2204d8640 1 -- 172.21.15.40:0/4123483283 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb1fc066440 msgr2=0x7fb1fc068900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:31.295 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:31.292+0000 7fb2204d8640 1 --2- 172.21.15.40:0/4123483283 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb1fc066440 0x7fb1fc068900 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fb21806d520 tx=0x7fb21400d040 comp rx=0 tx=0).stop 2023-10-16T08:38:31.295 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:31.292+0000 7fb2204d8640 1 -- 172.21.15.40:0/4123483283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb218156a20 msgr2=0x7fb218152240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:31.296 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:31.292+0000 7fb2204d8640 1 --2- 172.21.15.40:0/4123483283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb218156a20 0x7fb218152240 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fb210023a60 tx=0x7fb210022150 comp rx=0 tx=0).stop 2023-10-16T08:38:31.296 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:31.292+0000 7fb2204d8640 1 -- 172.21.15.40:0/4123483283 shutdown_connections 2023-10-16T08:38:31.296 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:31.292+0000 7fb2204d8640 1 --2- 172.21.15.40:0/4123483283 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb1fc066440 0x7fb1fc068900 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:31.296 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:31.292+0000 7fb2204d8640 1 --2- 172.21.15.40:0/4123483283 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb218156a20 0x7fb218152240 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:31.296 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:31.292+0000 7fb2204d8640 1 --2- 172.21.15.40:0/4123483283 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2180cb370 0x7fb218151d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:31.297 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:31.292+0000 7fb2204d8640 1 --2- 172.21.15.40:0/4123483283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2180adb80 0x7fb218151370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:31.297 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:31.292+0000 7fb2204d8640 1 -- 172.21.15.40:0/4123483283 >> 172.21.15.40:0/4123483283 conn(0x7fb2180b0470 msgr2=0x7fb218157000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:31.297 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:31.292+0000 7fb2204d8640 1 -- 172.21.15.40:0/4123483283 shutdown_connections 2023-10-16T08:38:31.297 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:31.292+0000 7fb2204d8640 1 -- 172.21.15.40:0/4123483283 wait complete. 2023-10-16T08:38:31.298 INFO:tasks.ceph.osd.6.smithi086.stderr:2023-10-16T08:38:31.297+0000 7f13858df640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T08:38:31.301 INFO:tasks.ceph.osd.4.smithi086.stderr:2023-10-16T08:38:31.301+0000 7f39d7261640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T08:38:31.302 INFO:tasks.ceph.osd.0.smithi040.stderr:2023-10-16T08:38:31.300+0000 7f9588c2a640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T08:38:31.310 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-16T08:38:31.355 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-10-16T08:38:31.352+0000 7f7f3ae8f640 -1 osd.1 0 waiting for initial osdmap 2023-10-16T08:38:31.357 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-10-16T08:38:31.352+0000 7fa078f90640 -1 osd.3 0 waiting for initial osdmap 2023-10-16T08:38:31.394 INFO:tasks.ceph.osd.1.smithi040.stderr:2023-10-16T08:38:31.392+0000 7f7f334d6640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T08:38:31.398 INFO:tasks.ceph.osd.3.smithi040.stderr:2023-10-16T08:38:31.396+0000 7fa0715d7640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-16T08:38:38.117 DEBUG:teuthology.orchestra.run.smithi040:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-16T08:38:38.243 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.240+0000 7efd984c2640 1 Processor -- start 2023-10-16T08:38:38.243 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.240+0000 7efd984c2640 1 -- start start 2023-10-16T08:38:38.244 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.240+0000 7efd984c2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd9006d440 0x7efd9006c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:38.245 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.240+0000 7efd984c2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efd90066190 0x7efd90160cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:38.245 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.240+0000 7efd984c2640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd9006ca90 con 0x7efd90070bd0 2023-10-16T08:38:38.245 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.240+0000 7efd984c2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd9006d820 con 0x7efd90066190 2023-10-16T08:38:38.245 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.240+0000 7efd984c2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd900cedc0 con 0x7efd9006d440 2023-10-16T08:38:38.245 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.240+0000 7efd96a5f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efd90066190 0x7efd90160cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:38.246 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.240+0000 7efd96a5f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efd90066190 0x7efd90160cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46534/0 (socket says 172.21.15.40:46534) 2023-10-16T08:38:38.246 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.240+0000 7efd96a5f640 1 -- 172.21.15.40:0/1890973577 learned_addr learned my addr 172.21.15.40:0/1890973577 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:38.246 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.240+0000 7efd9625e640 1 --2- 172.21.15.40:0/1890973577 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd9006d440 0x7efd9006c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:38.246 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.240+0000 7efd96a5f640 1 -- 172.21.15.40:0/1890973577 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd9006d440 msgr2=0x7efd9006c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:38.246 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.240+0000 7efd96a5f640 1 --2- 172.21.15.40:0/1890973577 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd9006d440 0x7efd9006c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.246 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.240+0000 7efd96a5f640 1 -- 172.21.15.40:0/1890973577 >> v1:172.21.15.86:6789/0 conn(0x7efd90070bd0 legacy=0x7efd90065a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:38.247 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.240+0000 7efd96a5f640 1 -- 172.21.15.40:0/1890973577 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd900cf0a0 con 0x7efd90066190 2023-10-16T08:38:38.247 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.244+0000 7efd96a5f640 1 --2- 172.21.15.40:0/1890973577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd90066190 0x7efd90160cd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efd88009e50 tx=0x7efd8801c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec3c628b301c841 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:38.247 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.244+0000 7efd9525c640 1 -- 172.21.15.40:0/1890973577 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd880a57e0 con 0x7efd90066190 2023-10-16T08:38:38.248 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.244+0000 7efd9525c640 1 -- 172.21.15.40:0/1890973577 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd880a5940 con 0x7efd90066190 2023-10-16T08:38:38.248 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.244+0000 7efd9525c640 1 -- 172.21.15.40:0/1890973577 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd880a5cd0 con 0x7efd90066190 2023-10-16T08:38:38.249 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.244+0000 7efd984c2640 1 -- 172.21.15.40:0/1890973577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd90066190 msgr2=0x7efd90160cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:38.249 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.244+0000 7efd984c2640 1 --2- 172.21.15.40:0/1890973577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd90066190 0x7efd90160cd0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7efd88009e50 tx=0x7efd8801c9e0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.249 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.244+0000 7efd984c2640 1 -- 172.21.15.40:0/1890973577 shutdown_connections 2023-10-16T08:38:38.249 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.244+0000 7efd984c2640 1 --2- 172.21.15.40:0/1890973577 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd90066190 0x7efd90160cd0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.249 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.244+0000 7efd984c2640 1 --2- 172.21.15.40:0/1890973577 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd9006d440 0x7efd9006c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.250 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.244+0000 7efd984c2640 1 -- 172.21.15.40:0/1890973577 >> 172.21.15.40:0/1890973577 conn(0x7efd900cc410 msgr2=0x7efd900cc820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:38.250 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.244+0000 7efd984c2640 1 -- 172.21.15.40:0/1890973577 shutdown_connections 2023-10-16T08:38:38.250 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.244+0000 7efd984c2640 1 -- 172.21.15.40:0/1890973577 wait complete. 2023-10-16T08:38:38.252 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.244+0000 7efd984c2640 1 Processor -- start 2023-10-16T08:38:38.252 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.244+0000 7efd984c2640 1 -- start start 2023-10-16T08:38:38.252 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.244+0000 7efd984c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd90066190 0x7efd9015bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:38.253 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.244+0000 7efd984c2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd9006d440 0x7efd90153ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:38.253 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd984c2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd90070bd0 0x7efd900c6a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:38.253 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd984c2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd900af000 con 0x7efd90070bd0 2023-10-16T08:38:38.253 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd984c2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd900afb80 con 0x7efd90066190 2023-10-16T08:38:38.254 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd984c2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd900af250 con 0x7efd9006d440 2023-10-16T08:38:38.254 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd96a5f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd90070bd0 0x7efd900c6a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:38.254 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd96a5f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd90070bd0 0x7efd900c6a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46536/0 (socket says 172.21.15.40:46536) 2023-10-16T08:38:38.254 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd96a5f640 1 -- 172.21.15.40:0/3531673240 learned_addr learned my addr 172.21.15.40:0/3531673240 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:38.254 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd95a5d640 1 --2- 172.21.15.40:0/3531673240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd9006d440 0x7efd90153ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:38.255 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd9625e640 1 --2- 172.21.15.40:0/3531673240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd90066190 0x7efd9015bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:38.255 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd96a5f640 1 -- 172.21.15.40:0/3531673240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd9006d440 msgr2=0x7efd90153ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:38.255 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd96a5f640 1 --2- 172.21.15.40:0/3531673240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd9006d440 0x7efd90153ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.255 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd96a5f640 1 -- 172.21.15.40:0/3531673240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd90066190 msgr2=0x7efd9015bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:38.255 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd96a5f640 1 --2- 172.21.15.40:0/3531673240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd90066190 0x7efd9015bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.256 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd96a5f640 1 -- 172.21.15.40:0/3531673240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd88009b30 con 0x7efd90070bd0 2023-10-16T08:38:38.256 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd96a5f640 1 --2- 172.21.15.40:0/3531673240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd90070bd0 0x7efd900c6a60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efd880af040 tx=0x7efd88019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:38.256 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd86ffd640 1 -- 172.21.15.40:0/3531673240 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd880bd070 con 0x7efd90070bd0 2023-10-16T08:38:38.256 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd86ffd640 1 -- 172.21.15.40:0/3531673240 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd880b8470 con 0x7efd90070bd0 2023-10-16T08:38:38.256 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd984c2640 1 -- 172.21.15.40:0/3531673240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd900af530 con 0x7efd90070bd0 2023-10-16T08:38:38.256 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd86ffd640 1 -- 172.21.15.40:0/3531673240 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd880b8840 con 0x7efd90070bd0 2023-10-16T08:38:38.257 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd984c2640 1 -- 172.21.15.40:0/3531673240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd9015c300 con 0x7efd90070bd0 2023-10-16T08:38:38.257 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd984c2640 1 -- 172.21.15.40:0/3531673240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd900afcf0 con 0x7efd90070bd0 2023-10-16T08:38:38.258 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd86ffd640 1 -- 172.21.15.40:0/3531673240 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efd880039a0 con 0x7efd90070bd0 2023-10-16T08:38:38.258 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd86ffd640 1 --2- 172.21.15.40:0/3531673240 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7efd740664d0 0x7efd74068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:38.258 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd86ffd640 1 -- 172.21.15.40:0/3531673240 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efd8811e550 con 0x7efd90070bd0 2023-10-16T08:38:38.258 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.248+0000 7efd9625e640 1 --2- 172.21.15.40:0/3531673240 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7efd740664d0 0x7efd74068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:38.259 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.252+0000 7efd9625e640 1 --2- 172.21.15.40:0/3531673240 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7efd740664d0 0x7efd74068990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00b930 tx=0x7efd8c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:38.259 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.252+0000 7efd86ffd640 1 -- 172.21.15.40:0/3531673240 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7efd880ebed0 con 0x7efd90070bd0 2023-10-16T08:38:38.417 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.412+0000 7efd984c2640 1 -- 172.21.15.40:0/3531673240 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efd9004e9c0 con 0x7efd90070bd0 2023-10-16T08:38:38.419 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.416+0000 7efd86ffd640 1 -- 172.21.15.40:0/3531673240 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12790 (secure 0 0 0) 0x7efd880ef4f0 con 0x7efd90070bd0 2023-10-16T08:38:38.420 INFO:teuthology.misc.health.smithi040.stdout: 2023-10-16T08:38:38.420 INFO:teuthology.misc.health.smithi040.stdout:{"epoch":16,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:35.290442+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:38:38.422 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.416+0000 7efd984c2640 1 -- 172.21.15.40:0/3531673240 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7efd740664d0 msgr2=0x7efd74068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:38.422 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.416+0000 7efd984c2640 1 --2- 172.21.15.40:0/3531673240 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7efd740664d0 0x7efd74068990 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00b930 tx=0x7efd8c008040 comp rx=0 tx=0).stop 2023-10-16T08:38:38.422 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.416+0000 7efd984c2640 1 -- 172.21.15.40:0/3531673240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd90070bd0 msgr2=0x7efd900c6a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:38.423 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.416+0000 7efd984c2640 1 --2- 172.21.15.40:0/3531673240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd90070bd0 0x7efd900c6a60 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7efd880af040 tx=0x7efd88019780 comp rx=0 tx=0).stop 2023-10-16T08:38:38.423 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.416+0000 7efd984c2640 1 -- 172.21.15.40:0/3531673240 shutdown_connections 2023-10-16T08:38:38.423 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.416+0000 7efd984c2640 1 --2- 172.21.15.40:0/3531673240 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7efd740664d0 0x7efd74068990 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.423 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.416+0000 7efd984c2640 1 --2- 172.21.15.40:0/3531673240 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efd90070bd0 0x7efd900c6a60 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.423 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.416+0000 7efd984c2640 1 --2- 172.21.15.40:0/3531673240 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efd9006d440 0x7efd90153ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.424 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.416+0000 7efd984c2640 1 --2- 172.21.15.40:0/3531673240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd90066190 0x7efd9015bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.424 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.416+0000 7efd984c2640 1 -- 172.21.15.40:0/3531673240 >> 172.21.15.40:0/3531673240 conn(0x7efd900cc410 msgr2=0x7efd900ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:38.424 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.416+0000 7efd984c2640 1 -- 172.21.15.40:0/3531673240 shutdown_connections 2023-10-16T08:38:38.424 INFO:teuthology.misc.health.smithi040.stderr:2023-10-16T08:38:38.416+0000 7efd984c2640 1 -- 172.21.15.40:0/3531673240 wait complete. 2023-10-16T08:38:38.438 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-16T08:38:38.438 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-16T08:38:38.439 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-16T08:38:38.439 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-16T08:38:38.441 INFO:tasks.ceph.mds.a:Started 2023-10-16T08:38:38.442 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-16T08:38:38.442 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-16T08:38:38.445 INFO:tasks.ceph.mds.c:Started 2023-10-16T08:38:38.445 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-16T08:38:38.445 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-16T08:38:38.448 INFO:tasks.ceph.mds.b:Started 2023-10-16T08:38:38.448 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-16T08:38:38.449 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-16T08:38:38.451 INFO:tasks.ceph.mds.d:Started 2023-10-16T08:38:38.451 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-16T08:38:38.452 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:38:38.528 INFO:tasks.ceph.mds.b.smithi086.stdout:starting mds.b at 2023-10-16T08:38:38.528 INFO:tasks.ceph.mds.d.smithi086.stdout:starting mds.d at 2023-10-16T08:38:38.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.564+0000 7f103b7f4640 1 Processor -- start 2023-10-16T08:38:38.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.564+0000 7f103b7f4640 1 -- start start 2023-10-16T08:38:38.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.564+0000 7f103b7f4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340c6dc0 0x7f10340cb050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:38.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.564+0000 7f103b7f4640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f10340c71a0 0x7f10340b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:38.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.564+0000 7f103b7f4640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10340d0680 con 0x7f10340b29c0 2023-10-16T08:38:38.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.564+0000 7f103b7f4640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10340b2e50 con 0x7f10340c71a0 2023-10-16T08:38:38.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.564+0000 7f103b7f4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10340cf6f0 con 0x7f10340c6dc0 2023-10-16T08:38:38.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.568+0000 7f1038d8f640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f10340c71a0 0x7f10340b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:38.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.568+0000 7f1039590640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340c6dc0 0x7f10340cb050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:38.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.568+0000 7f1039d91640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f10340b29c0 0x7f103415ec40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.40:50694/0 (socket says 172.21.15.40:50694) 2023-10-16T08:38:38.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.568+0000 7f1039d91640 1 -- 172.21.15.40:0/1430374647 learned_addr learned my addr 172.21.15.40:0/1430374647 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:38.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.568+0000 7f1038d8f640 1 -- 172.21.15.40:0/1430374647 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340c6dc0 msgr2=0x7f10340cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:38.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.568+0000 7f1038d8f640 1 --2- 172.21.15.40:0/1430374647 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340c6dc0 0x7f10340cb050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.568+0000 7f1038d8f640 1 -- 172.21.15.40:0/1430374647 >> v1:172.21.15.86:6789/0 conn(0x7f10340b29c0 legacy=0x7f103415ec40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:38.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.568+0000 7f1038d8f640 1 -- 172.21.15.40:0/1430374647 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10340cf9d0 con 0x7f10340c71a0 2023-10-16T08:38:38.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.568+0000 7f102bfff640 1 -- 172.21.15.40:0/1430374647 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3967450126 0 0) 0x7f10340d0680 con 0x7f10340b29c0 2023-10-16T08:38:38.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.568+0000 7f1039590640 1 --2- 172.21.15.40:0/1430374647 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340c6dc0 0x7f10340cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:38:38.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.568+0000 7f1038d8f640 1 --2- 172.21.15.40:0/1430374647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10340c71a0 0x7f10340b2480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f10240097c0 tx=0x7f102401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f5b30ea5a64d729 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:38.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.568+0000 7f102bfff640 1 -- 172.21.15.40:0/1430374647 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10240ab650 con 0x7f10340c71a0 2023-10-16T08:38:38.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.568+0000 7f102bfff640 1 -- 172.21.15.40:0/1430374647 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10240ab7b0 con 0x7f10340c71a0 2023-10-16T08:38:38.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.568+0000 7f102bfff640 1 -- 172.21.15.40:0/1430374647 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10240abb40 con 0x7f10340c71a0 2023-10-16T08:38:38.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.572+0000 7f103b7f4640 1 -- 172.21.15.40:0/1430374647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10340c71a0 msgr2=0x7f10340b2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:38.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.572+0000 7f103b7f4640 1 --2- 172.21.15.40:0/1430374647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10340c71a0 0x7f10340b2480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f10240097c0 tx=0x7f102401c450 comp rx=0 tx=0).stop 2023-10-16T08:38:38.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.572+0000 7f103b7f4640 1 -- 172.21.15.40:0/1430374647 shutdown_connections 2023-10-16T08:38:38.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.572+0000 7f103b7f4640 1 --2- 172.21.15.40:0/1430374647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10340c71a0 0x7f10340b2480 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.572+0000 7f103b7f4640 1 --2- 172.21.15.40:0/1430374647 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340c6dc0 0x7f10340cb050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.572+0000 7f103b7f4640 1 -- 172.21.15.40:0/1430374647 >> 172.21.15.40:0/1430374647 conn(0x7f103406d700 msgr2=0x7f103406db10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.572+0000 7f103b7f4640 1 -- 172.21.15.40:0/1430374647 shutdown_connections 2023-10-16T08:38:38.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.572+0000 7f103b7f4640 1 -- 172.21.15.40:0/1430374647 wait complete. 2023-10-16T08:38:38.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f103b7f4640 1 Processor -- start 2023-10-16T08:38:38.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f103b7f4640 1 -- start start 2023-10-16T08:38:38.579 INFO:tasks.ceph.mds.a.smithi040.stdout:starting mds.a at 2023-10-16T08:38:38.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f103b7f4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10340b29c0 0x7f103413d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:38.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f103b7f4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10340c6dc0 0x7f10341f71b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:38.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f103b7f4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340c71a0 0x7f10340b5330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:38.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f103b7f4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10341f76f0 con 0x7f10340c6dc0 2023-10-16T08:38:38.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f103b7f4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10340b3680 con 0x7f10340b29c0 2023-10-16T08:38:38.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f103b7f4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103413db40 con 0x7f10340c71a0 2023-10-16T08:38:38.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f1039590640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10340b29c0 0x7f103413d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:38.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f1039590640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10340b29c0 0x7f103413d4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.40:43964/0 (socket says 172.21.15.40:43964) 2023-10-16T08:38:38.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f1039590640 1 -- 172.21.15.40:0/3113827026 learned_addr learned my addr 172.21.15.40:0/3113827026 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:38.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f1039d91640 1 --2- 172.21.15.40:0/3113827026 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340c71a0 0x7f10340b5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:38.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f1038d8f640 1 --2- 172.21.15.40:0/3113827026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10340c6dc0 0x7f10341f71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:38.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f1039590640 1 -- 172.21.15.40:0/3113827026 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340c71a0 msgr2=0x7f10340b5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:38.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f1039590640 1 --2- 172.21.15.40:0/3113827026 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340c71a0 0x7f10340b5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f1039590640 1 -- 172.21.15.40:0/3113827026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10340c6dc0 msgr2=0x7f10341f71b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:38.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f1039590640 1 --2- 172.21.15.40:0/3113827026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10340c6dc0 0x7f10341f71b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f1039590640 1 -- 172.21.15.40:0/3113827026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1030009420 con 0x7f10340b29c0 2023-10-16T08:38:38.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f1039590640 1 --2- 172.21.15.40:0/3113827026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10340b29c0 0x7f103413d4f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f103000ebe0 tx=0x7f103000c520 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:38.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f1029ffb640 1 -- 172.21.15.40:0/3113827026 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103000eee0 con 0x7f10340b29c0 2023-10-16T08:38:38.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f1029ffb640 1 -- 172.21.15.40:0/3113827026 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103000cc30 con 0x7f10340b29c0 2023-10-16T08:38:38.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f103b7f4640 1 -- 172.21.15.40:0/3113827026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1024009420 con 0x7f10340b29c0 2023-10-16T08:38:38.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f1029ffb640 1 -- 172.21.15.40:0/3113827026 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1030013470 con 0x7f10340b29c0 2023-10-16T08:38:38.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.576+0000 7f103b7f4640 1 -- 172.21.15.40:0/3113827026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f103413e100 con 0x7f10340b29c0 2023-10-16T08:38:38.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.580+0000 7f103b7f4640 1 -- 172.21.15.40:0/3113827026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f103404e9c0 con 0x7f10340b29c0 2023-10-16T08:38:38.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.580+0000 7f1029ffb640 1 -- 172.21.15.40:0/3113827026 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f10300135d0 con 0x7f10340b29c0 2023-10-16T08:38:38.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.580+0000 7f1029ffb640 1 --2- 172.21.15.40:0/3113827026 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f10180664a0 0x7f1018068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:38.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.580+0000 7f1029ffb640 1 -- 172.21.15.40:0/3113827026 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f1030083da0 con 0x7f10340b29c0 2023-10-16T08:38:38.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.580+0000 7f1038d8f640 1 --2- 172.21.15.40:0/3113827026 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f10180664a0 0x7f1018068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:38.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.580+0000 7f1038d8f640 1 --2- 172.21.15.40:0/3113827026 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f10180664a0 0x7f1018068960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f10240af000 tx=0x7f10240a7000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:38.588 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-10-16T08:38:38.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.584+0000 7f1029ffb640 1 -- 172.21.15.40:0/3113827026 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f1030051600 con 0x7f10340b29c0 2023-10-16T08:38:38.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.744+0000 7f103b7f4640 1 -- 172.21.15.40:0/3113827026 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f103406f230 con 0x7f10340b29c0 2023-10-16T08:38:38.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.748+0000 7f1029ffb640 1 -- 172.21.15.40:0/3113827026 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12790 (secure 0 0 0) 0x7f1030054c20 con 0x7f10340b29c0 2023-10-16T08:38:38.751 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:38.751 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:35.290442+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:38:38.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.748+0000 7f103b7f4640 1 -- 172.21.15.40:0/3113827026 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f10180664a0 msgr2=0x7f1018068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:38.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.748+0000 7f103b7f4640 1 --2- 172.21.15.40:0/3113827026 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f10180664a0 0x7f1018068960 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f10240af000 tx=0x7f10240a7000 comp rx=0 tx=0).stop 2023-10-16T08:38:38.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.748+0000 7f103b7f4640 1 -- 172.21.15.40:0/3113827026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10340b29c0 msgr2=0x7f103413d4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:38.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.748+0000 7f103b7f4640 1 --2- 172.21.15.40:0/3113827026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10340b29c0 0x7f103413d4f0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f103000ebe0 tx=0x7f103000c520 comp rx=0 tx=0).stop 2023-10-16T08:38:38.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.748+0000 7f103b7f4640 1 -- 172.21.15.40:0/3113827026 shutdown_connections 2023-10-16T08:38:38.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.748+0000 7f103b7f4640 1 --2- 172.21.15.40:0/3113827026 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10340c71a0 0x7f10340b5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.748+0000 7f103b7f4640 1 --2- 172.21.15.40:0/3113827026 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f10180664a0 0x7f1018068960 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.748+0000 7f103b7f4640 1 --2- 172.21.15.40:0/3113827026 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10340c6dc0 0x7f10341f71b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.748+0000 7f103b7f4640 1 --2- 172.21.15.40:0/3113827026 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10340b29c0 0x7f103413d4f0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.748+0000 7f103b7f4640 1 -- 172.21.15.40:0/3113827026 >> 172.21.15.40:0/3113827026 conn(0x7f103406d700 msgr2=0x7f10340b1ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.748+0000 7f103b7f4640 1 -- 172.21.15.40:0/3113827026 shutdown_connections 2023-10-16T08:38:38.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.748+0000 7f103b7f4640 1 -- 172.21.15.40:0/3113827026 wait complete. 2023-10-16T08:38:38.770 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:38:33.014668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:38:38.771 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:38:38.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.896+0000 7f68569e8640 1 Processor -- start 2023-10-16T08:38:38.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.896+0000 7f68569e8640 1 -- start start 2023-10-16T08:38:38.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.896+0000 7f68569e8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f68500cde70 0x7f685006d340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:38.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.896+0000 7f68569e8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685006c580 con 0x7f68500b19e0 2023-10-16T08:38:38.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.896+0000 7f68569e8640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f685006c860 con 0x7f68500ce250 2023-10-16T08:38:38.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.896+0000 7f68569e8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685006cb40 con 0x7f68500cde70 2023-10-16T08:38:38.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.896+0000 7f684ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f68500cde70 0x7f685006d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:38.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.896+0000 7f684ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f68500cde70 0x7f685006d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46572/0 (socket says 172.21.15.40:46572) 2023-10-16T08:38:38.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.896+0000 7f684ffff640 1 -- 172.21.15.40:0/1349870914 learned_addr learned my addr 172.21.15.40:0/1349870914 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:38.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.900+0000 7f684ffff640 1 -- 172.21.15.40:0/1349870914 >> v1:172.21.15.86:6789/0 conn(0x7f68500ce250 legacy=0x7f68500b12d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:38.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.900+0000 7f684ffff640 1 -- 172.21.15.40:0/1349870914 >> v1:172.21.15.40:6789/0 conn(0x7f68500b19e0 legacy=0x7f68501594a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:38.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.900+0000 7f684ffff640 1 -- 172.21.15.40:0/1349870914 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f685006ce20 con 0x7f68500cde70 2023-10-16T08:38:38.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.900+0000 7f684ffff640 1 --2- 172.21.15.40:0/1349870914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68500cde70 0x7f685006d340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6840009870 tx=0x7f684001e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64d9c79098481e0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:38.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.900+0000 7f684effd640 1 -- 172.21.15.40:0/1349870914 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400a8700 con 0x7f68500cde70 2023-10-16T08:38:38.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.900+0000 7f684effd640 1 -- 172.21.15.40:0/1349870914 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68400a8860 con 0x7f68500cde70 2023-10-16T08:38:38.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.900+0000 7f684effd640 1 -- 172.21.15.40:0/1349870914 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400a8bf0 con 0x7f68500cde70 2023-10-16T08:38:38.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.900+0000 7f68569e8640 1 -- 172.21.15.40:0/1349870914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68500cde70 msgr2=0x7f685006d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:38.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.900+0000 7f68569e8640 1 --2- 172.21.15.40:0/1349870914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68500cde70 0x7f685006d340 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f6840009870 tx=0x7f684001e710 comp rx=0 tx=0).stop 2023-10-16T08:38:38.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.900+0000 7f68569e8640 1 -- 172.21.15.40:0/1349870914 shutdown_connections 2023-10-16T08:38:38.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.900+0000 7f68569e8640 1 --2- 172.21.15.40:0/1349870914 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68500cde70 0x7f685006d340 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.900+0000 7f68569e8640 1 -- 172.21.15.40:0/1349870914 >> 172.21.15.40:0/1349870914 conn(0x7f68500cc2d0 msgr2=0x7f68500cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:38.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.900+0000 7f68569e8640 1 -- 172.21.15.40:0/1349870914 shutdown_connections 2023-10-16T08:38:38.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.900+0000 7f68569e8640 1 -- 172.21.15.40:0/1349870914 wait complete. 2023-10-16T08:38:38.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f68569e8640 1 Processor -- start 2023-10-16T08:38:38.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f68569e8640 1 -- start start 2023-10-16T08:38:38.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f68569e8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68500b19e0 0x7f685015b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:38.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f68569e8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68500cde70 0x7f685015c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:38.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f68569e8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68500ce250 0x7f68500c5660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:38.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f68569e8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685015cbd0 con 0x7f68500b19e0 2023-10-16T08:38:38.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f68569e8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f685015b5f0 con 0x7f68500ce250 2023-10-16T08:38:38.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f68569e8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f685015b8d0 con 0x7f68500cde70 2023-10-16T08:38:38.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f684ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68500b19e0 0x7f685015b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:38.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f684ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68500b19e0 0x7f685015b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46574/0 (socket says 172.21.15.40:46574) 2023-10-16T08:38:38.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f684ffff640 1 -- 172.21.15.40:0/823481270 learned_addr learned my addr 172.21.15.40:0/823481270 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:38.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f684f7fe640 1 --2- 172.21.15.40:0/823481270 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68500cde70 0x7f685015c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:38.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f6854f85640 1 --2- 172.21.15.40:0/823481270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68500ce250 0x7f68500c5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:38.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f684ffff640 1 -- 172.21.15.40:0/823481270 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68500cde70 msgr2=0x7f685015c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:38.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f684ffff640 1 --2- 172.21.15.40:0/823481270 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68500cde70 0x7f685015c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f684ffff640 1 -- 172.21.15.40:0/823481270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68500ce250 msgr2=0x7f68500c5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:38.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f684ffff640 1 --2- 172.21.15.40:0/823481270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68500ce250 0x7f68500c5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:38.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f684ffff640 1 -- 172.21.15.40:0/823481270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68400094d0 con 0x7f68500b19e0 2023-10-16T08:38:38.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f684f7fe640 1 --2- 172.21.15.40:0/823481270 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68500cde70 0x7f685015c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:38:38.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f684ffff640 1 --2- 172.21.15.40:0/823481270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68500b19e0 0x7f685015b010 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f68400b2040 tx=0x7f684001cf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:38.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f684cff9640 1 -- 172.21.15.40:0/823481270 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400b0c60 con 0x7f68500b19e0 2023-10-16T08:38:38.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f684cff9640 1 -- 172.21.15.40:0/823481270 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68400b0de0 con 0x7f68500b19e0 2023-10-16T08:38:38.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f68569e8640 1 -- 172.21.15.40:0/823481270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f685015bbb0 con 0x7f68500b19e0 2023-10-16T08:38:38.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f684cff9640 1 -- 172.21.15.40:0/823481270 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68400ba670 con 0x7f68500b19e0 2023-10-16T08:38:38.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f68569e8640 1 -- 172.21.15.40:0/823481270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685015c170 con 0x7f68500b19e0 2023-10-16T08:38:38.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.904+0000 7f68569e8640 1 -- 172.21.15.40:0/823481270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685015c360 con 0x7f68500b19e0 2023-10-16T08:38:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.908+0000 7f684cff9640 1 -- 172.21.15.40:0/823481270 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f68400ba7d0 con 0x7f68500b19e0 2023-10-16T08:38:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.908+0000 7f684cff9640 1 --2- 172.21.15.40:0/823481270 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f683c0664a0 0x7f683c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:38.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.908+0000 7f684cff9640 1 -- 172.21.15.40:0/823481270 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f68400ac070 con 0x7f68500b19e0 2023-10-16T08:38:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.908+0000 7f684f7fe640 1 --2- 172.21.15.40:0/823481270 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f683c0664a0 0x7f683c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.908+0000 7f684f7fe640 1 --2- 172.21.15.40:0/823481270 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f683c0664a0 0x7f683c068960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f68500ae620 tx=0x7f6844008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:38.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:38.912+0000 7f684cff9640 1 -- 172.21.15.40:0/823481270 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f68400ecaf0 con 0x7f68500b19e0 2023-10-16T08:38:39.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.064+0000 7f68569e8640 1 -- 172.21.15.40:0/823481270 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f685004e9c0 con 0x7f68500b19e0 2023-10-16T08:38:39.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.068+0000 7f684cff9640 1 -- 172.21.15.40:0/823481270 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f68400f0110 con 0x7f68500b19e0 2023-10-16T08:38:39.071 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-16T08:38:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.068+0000 7f68569e8640 1 -- 172.21.15.40:0/823481270 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f683c0664a0 msgr2=0x7f683c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.068+0000 7f68569e8640 1 --2- 172.21.15.40:0/823481270 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f683c0664a0 0x7f683c068960 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f68500ae620 tx=0x7f6844008040 comp rx=0 tx=0).stop 2023-10-16T08:38:39.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.068+0000 7f68569e8640 1 -- 172.21.15.40:0/823481270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68500b19e0 msgr2=0x7f685015b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:39.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.068+0000 7f68569e8640 1 --2- 172.21.15.40:0/823481270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68500b19e0 0x7f685015b010 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f68400b2040 tx=0x7f684001cf50 comp rx=0 tx=0).stop 2023-10-16T08:38:39.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.068+0000 7f68569e8640 1 -- 172.21.15.40:0/823481270 shutdown_connections 2023-10-16T08:38:39.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.068+0000 7f68569e8640 1 --2- 172.21.15.40:0/823481270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68500ce250 0x7f68500c5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:39.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.068+0000 7f68569e8640 1 --2- 172.21.15.40:0/823481270 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f68500cde70 0x7f685015c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:39.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.068+0000 7f68569e8640 1 --2- 172.21.15.40:0/823481270 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f683c0664a0 0x7f683c068960 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:39.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.068+0000 7f68569e8640 1 --2- 172.21.15.40:0/823481270 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f68500b19e0 0x7f685015b010 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:39.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.068+0000 7f68569e8640 1 -- 172.21.15.40:0/823481270 >> 172.21.15.40:0/823481270 conn(0x7f68500cc2d0 msgr2=0x7f68500cc740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:39.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.068+0000 7f68569e8640 1 -- 172.21.15.40:0/823481270 shutdown_connections 2023-10-16T08:38:39.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.068+0000 7f68569e8640 1 -- 172.21.15.40:0/823481270 wait complete. 2023-10-16T08:38:39.089 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:38:39.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.200+0000 7f667fb5d640 1 Processor -- start 2023-10-16T08:38:39.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.200+0000 7f667fb5d640 1 -- start start 2023-10-16T08:38:39.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.204+0000 7f667fb5d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f66780cab70 0x7f66780caf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:39.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.204+0000 7f667fb5d640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f667815e790 0x7f66780cb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:39.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.204+0000 7f667fb5d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f667815b070 con 0x7f667815ac90 2023-10-16T08:38:39.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.204+0000 7f667fb5d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f667806d3b0 con 0x7f66780cab70 2023-10-16T08:38:39.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.204+0000 7f667fb5d640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f667806d650 con 0x7f667815e790 2023-10-16T08:38:39.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.204+0000 7f667d0f8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f66780cab70 0x7f66780caf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:39.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.204+0000 7f667d8f9640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f667815ac90 0x7f66780712f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60614/0 (socket says 172.21.15.40:60614) 2023-10-16T08:38:39.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.204+0000 7f667d8f9640 1 -- 172.21.15.40:0/2967348927 learned_addr learned my addr 172.21.15.40:0/2967348927 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:39.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.204+0000 7f667e0fa640 1 --2- 172.21.15.40:0/2967348927 >> v2:172.21.15.86:3300/0 conn(0x7f667815e790 0x7f66780cb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:39.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.204+0000 7f667d0f8640 1 -- 172.21.15.40:0/2967348927 >> v2:172.21.15.86:3300/0 conn(0x7f667815e790 msgr2=0x7f66780cb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:39.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.204+0000 7f667d0f8640 1 --2- 172.21.15.40:0/2967348927 >> v2:172.21.15.86:3300/0 conn(0x7f667815e790 0x7f66780cb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:39.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.204+0000 7f667d0f8640 1 -- 172.21.15.40:0/2967348927 >> v1:172.21.15.40:6789/0 conn(0x7f667815ac90 legacy=0x7f66780712f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:39.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.204+0000 7f667d0f8640 1 -- 172.21.15.40:0/2967348927 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667806d930 con 0x7f66780cab70 2023-10-16T08:38:39.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.204+0000 7f667e0fa640 1 --2- 172.21.15.40:0/2967348927 >> v2:172.21.15.86:3300/0 conn(0x7f667815e790 0x7f66780cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:39.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.204+0000 7f667c8f7640 1 -- 172.21.15.40:0/2967348927 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 599807228 0 0) 0x7f667815b070 con 0x7f667815ac90 2023-10-16T08:38:39.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.204+0000 7f667d0f8640 1 --2- 172.21.15.40:0/2967348927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66780cab70 0x7f66780caf70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f66780ccbb0 tx=0x7f666801c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36a1e5e5a371a02d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:39.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.204+0000 7f667c8f7640 1 -- 172.21.15.40:0/2967348927 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66680ab600 con 0x7f66780cab70 2023-10-16T08:38:39.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.204+0000 7f667c8f7640 1 -- 172.21.15.40:0/2967348927 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66680a7050 con 0x7f66780cab70 2023-10-16T08:38:39.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.204+0000 7f667c8f7640 1 -- 172.21.15.40:0/2967348927 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66680ab980 con 0x7f66780cab70 2023-10-16T08:38:39.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667fb5d640 1 -- 172.21.15.40:0/2967348927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66780cab70 msgr2=0x7f66780caf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:39.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667fb5d640 1 --2- 172.21.15.40:0/2967348927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66780cab70 0x7f66780caf70 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f66780ccbb0 tx=0x7f666801c430 comp rx=0 tx=0).stop 2023-10-16T08:38:39.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667fb5d640 1 -- 172.21.15.40:0/2967348927 shutdown_connections 2023-10-16T08:38:39.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667fb5d640 1 --2- 172.21.15.40:0/2967348927 >> v2:172.21.15.86:3300/0 conn(0x7f667815e790 0x7f66780cb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:39.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667fb5d640 1 --2- 172.21.15.40:0/2967348927 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f66780cab70 0x7f66780caf70 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:39.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667fb5d640 1 -- 172.21.15.40:0/2967348927 >> 172.21.15.40:0/2967348927 conn(0x7f66780b0be0 msgr2=0x7f66780b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:39.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667fb5d640 1 -- 172.21.15.40:0/2967348927 shutdown_connections 2023-10-16T08:38:39.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667fb5d640 1 -- 172.21.15.40:0/2967348927 wait complete. 2023-10-16T08:38:39.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667fb5d640 1 Processor -- start 2023-10-16T08:38:39.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667fb5d640 1 -- start start 2023-10-16T08:38:39.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667fb5d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66780cab70 0x7f6678152480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:39.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667fb5d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f667815ac90 0x7f66780cee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:39.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667fb5d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f667815e790 0x7f66780cf390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:39.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667fb5d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66780c5860 con 0x7f667815ac90 2023-10-16T08:38:39.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667fb5d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66780c4c50 con 0x7f667815e790 2023-10-16T08:38:39.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667fb5d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66780c4f30 con 0x7f66780cab70 2023-10-16T08:38:39.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667d0f8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f667815ac90 0x7f66780cee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:39.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667d0f8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f667815ac90 0x7f66780cee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46590/0 (socket says 172.21.15.40:46590) 2023-10-16T08:38:39.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667d0f8640 1 -- 172.21.15.40:0/1014571405 learned_addr learned my addr 172.21.15.40:0/1014571405 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:39.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667d8f9640 1 --2- 172.21.15.40:0/1014571405 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66780cab70 0x7f6678152480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:39.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667e0fa640 1 --2- 172.21.15.40:0/1014571405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f667815e790 0x7f66780cf390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:39.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667d0f8640 1 -- 172.21.15.40:0/1014571405 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66780cab70 msgr2=0x7f6678152480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:39.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667d0f8640 1 --2- 172.21.15.40:0/1014571405 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66780cab70 0x7f6678152480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:39.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667d0f8640 1 -- 172.21.15.40:0/1014571405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f667815e790 msgr2=0x7f66780cf390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:39.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667d0f8640 1 --2- 172.21.15.40:0/1014571405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f667815e790 0x7f66780cf390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:39.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667d0f8640 1 -- 172.21.15.40:0/1014571405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6668009420 con 0x7f667815ac90 2023-10-16T08:38:39.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667d0f8640 1 --2- 172.21.15.40:0/1014571405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f667815ac90 0x7f66780cee50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f666801a060 tx=0x7f666801ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:39.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f666e7fc640 1 -- 172.21.15.40:0/1014571405 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66680b37d0 con 0x7f667815ac90 2023-10-16T08:38:39.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f666e7fc640 1 -- 172.21.15.40:0/1014571405 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66680b3930 con 0x7f667815ac90 2023-10-16T08:38:39.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667fb5d640 1 -- 172.21.15.40:0/1014571405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66780c5210 con 0x7f667815ac90 2023-10-16T08:38:39.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f666e7fc640 1 -- 172.21.15.40:0/1014571405 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66680b3cc0 con 0x7f667815ac90 2023-10-16T08:38:39.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.208+0000 7f667fb5d640 1 -- 172.21.15.40:0/1014571405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6678153c70 con 0x7f667815ac90 2023-10-16T08:38:39.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.212+0000 7f667fb5d640 1 -- 172.21.15.40:0/1014571405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66780c5a10 con 0x7f667815ac90 2023-10-16T08:38:39.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.212+0000 7f666e7fc640 1 -- 172.21.15.40:0/1014571405 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f66680b2070 con 0x7f667815ac90 2023-10-16T08:38:39.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.212+0000 7f666e7fc640 1 --2- 172.21.15.40:0/1014571405 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f665c0664d0 0x7f665c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:39.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.212+0000 7f666e7fc640 1 -- 172.21.15.40:0/1014571405 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f666811de30 con 0x7f667815ac90 2023-10-16T08:38:39.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.212+0000 7f667d8f9640 1 --2- 172.21.15.40:0/1014571405 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f665c0664d0 0x7f665c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:39.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.212+0000 7f667d8f9640 1 --2- 172.21.15.40:0/1014571405 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f665c0664d0 0x7f665c068990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6674007fd0 tx=0x7f6674009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:39.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.216+0000 7f666e7fc640 1 -- 172.21.15.40:0/1014571405 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f66780c5a10 con 0x7f667815ac90 2023-10-16T08:38:39.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.400+0000 7f667fb5d640 1 -- 172.21.15.40:0/1014571405 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f667804e9c0 con 0x7f667815ac90 2023-10-16T08:38:39.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.404+0000 7f666e7fc640 1 -- 172.21.15.40:0/1014571405 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1318 (secure 0 0 0) 0x7f66680b0020 con 0x7f667815ac90 2023-10-16T08:38:39.407 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:39.407 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/32529848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":32529848},{"type":"v1","addr":"172.21.15.86:6837","nonce":32529848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2}],"filesystems":[]} 2023-10-16T08:38:39.407 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 2 2023-10-16T08:38:39.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.404+0000 7f667fb5d640 1 -- 172.21.15.40:0/1014571405 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f665c0664d0 msgr2=0x7f665c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:39.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.404+0000 7f667fb5d640 1 --2- 172.21.15.40:0/1014571405 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f665c0664d0 0x7f665c068990 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6674007fd0 tx=0x7f6674009040 comp rx=0 tx=0).stop 2023-10-16T08:38:39.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.404+0000 7f667fb5d640 1 -- 172.21.15.40:0/1014571405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f667815ac90 msgr2=0x7f66780cee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:39.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.404+0000 7f667fb5d640 1 --2- 172.21.15.40:0/1014571405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f667815ac90 0x7f66780cee50 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f666801a060 tx=0x7f666801ac70 comp rx=0 tx=0).stop 2023-10-16T08:38:39.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.404+0000 7f667fb5d640 1 -- 172.21.15.40:0/1014571405 shutdown_connections 2023-10-16T08:38:39.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.404+0000 7f667fb5d640 1 --2- 172.21.15.40:0/1014571405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f667815e790 0x7f66780cf390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:39.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.404+0000 7f667fb5d640 1 --2- 172.21.15.40:0/1014571405 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f665c0664d0 0x7f665c068990 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:39.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.404+0000 7f667fb5d640 1 --2- 172.21.15.40:0/1014571405 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f667815ac90 0x7f66780cee50 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:39.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.404+0000 7f667fb5d640 1 --2- 172.21.15.40:0/1014571405 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f66780cab70 0x7f6678152480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:39.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.404+0000 7f667fb5d640 1 -- 172.21.15.40:0/1014571405 >> 172.21.15.40:0/1014571405 conn(0x7f66780b0be0 msgr2=0x7f66780afaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:39.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.404+0000 7f667fb5d640 1 -- 172.21.15.40:0/1014571405 shutdown_connections 2023-10-16T08:38:39.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:39.404+0000 7f667fb5d640 1 -- 172.21.15.40:0/1014571405 wait complete. 2023-10-16T08:38:41.425 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:38:41.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.556+0000 7fcd6c01c640 1 Processor -- start 2023-10-16T08:38:41.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.556+0000 7fcd6c01c640 1 -- start start 2023-10-16T08:38:41.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.556+0000 7fcd6c01c640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fcd640cc320 0x7fcd640b28f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:41.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.556+0000 7fcd6c01c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcd640b4370 0x7fcd640715d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:41.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.556+0000 7fcd6c01c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd640b2e30 0x7fcd64071b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:41.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.556+0000 7fcd6c01c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd6415b130 con 0x7fcd640b4370 2023-10-16T08:38:41.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.556+0000 7fcd6c01c640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd640b3ee0 con 0x7fcd640cc320 2023-10-16T08:38:41.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.556+0000 7fcd6c01c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd64072050 con 0x7fcd640b2e30 2023-10-16T08:38:41.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.556+0000 7fcd695b7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcd640b4370 0x7fcd640715d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:41.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.556+0000 7fcd695b7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcd640b4370 0x7fcd640715d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46606/0 (socket says 172.21.15.40:46606) 2023-10-16T08:38:41.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.556+0000 7fcd695b7640 1 -- 172.21.15.40:0/1937228824 learned_addr learned my addr 172.21.15.40:0/1937228824 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:41.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.556+0000 7fcd6a5b9640 1 --2- 172.21.15.40:0/1937228824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd640b2e30 0x7fcd64071b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:41.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.556+0000 7fcd69db8640 1 --2- 172.21.15.40:0/1937228824 >> v2:172.21.15.86:3300/0 conn(0x7fcd640cc320 0x7fcd640b28f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:41.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.556+0000 7fcd695b7640 1 -- 172.21.15.40:0/1937228824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd640b2e30 msgr2=0x7fcd64071b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:41.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.556+0000 7fcd695b7640 1 --2- 172.21.15.40:0/1937228824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd640b2e30 0x7fcd64071b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:41.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.556+0000 7fcd695b7640 1 -- 172.21.15.40:0/1937228824 >> v2:172.21.15.86:3300/0 conn(0x7fcd640cc320 msgr2=0x7fcd640b28f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:41.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.556+0000 7fcd695b7640 1 --2- 172.21.15.40:0/1937228824 >> v2:172.21.15.86:3300/0 conn(0x7fcd640cc320 0x7fcd640b28f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:41.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.556+0000 7fcd695b7640 1 -- 172.21.15.40:0/1937228824 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd640b32d0 con 0x7fcd640b4370 2023-10-16T08:38:41.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd695b7640 1 --2- 172.21.15.40:0/1937228824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd640b4370 0x7fcd640715d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcd6406f410 tx=0x7fcd5401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c668f4868f7ade63 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:41.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd68db6640 1 -- 172.21.15.40:0/1937228824 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd540aa5f0 con 0x7fcd640b4370 2023-10-16T08:38:41.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd68db6640 1 -- 172.21.15.40:0/1937228824 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd540aa750 con 0x7fcd640b4370 2023-10-16T08:38:41.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd68db6640 1 -- 172.21.15.40:0/1937228824 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd540aaae0 con 0x7fcd640b4370 2023-10-16T08:38:41.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd6c01c640 1 -- 172.21.15.40:0/1937228824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd640b4370 msgr2=0x7fcd640715d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:41.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd6c01c640 1 --2- 172.21.15.40:0/1937228824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd640b4370 0x7fcd640715d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fcd6406f410 tx=0x7fcd5401c4d0 comp rx=0 tx=0).stop 2023-10-16T08:38:41.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd6c01c640 1 -- 172.21.15.40:0/1937228824 shutdown_connections 2023-10-16T08:38:41.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd6c01c640 1 --2- 172.21.15.40:0/1937228824 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd640b2e30 0x7fcd64071b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:41.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd6c01c640 1 --2- 172.21.15.40:0/1937228824 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd640b4370 0x7fcd640715d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:41.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd6c01c640 1 --2- 172.21.15.40:0/1937228824 >> v2:172.21.15.86:3300/0 conn(0x7fcd640cc320 0x7fcd640b28f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:41.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd6c01c640 1 -- 172.21.15.40:0/1937228824 >> 172.21.15.40:0/1937228824 conn(0x7fcd640b0be0 msgr2=0x7fcd640b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:41.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd6c01c640 1 -- 172.21.15.40:0/1937228824 shutdown_connections 2023-10-16T08:38:41.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd6c01c640 1 -- 172.21.15.40:0/1937228824 wait complete. 2023-10-16T08:38:41.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd6c01c640 1 Processor -- start 2023-10-16T08:38:41.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd6c01c640 1 -- start start 2023-10-16T08:38:41.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd6c01c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd640b2e30 0x7fcd640c59a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:41.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd6c01c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd640cc320 0x7fcd640c4060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:41.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd6c01c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd6413e080 0x7fcd6413e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:41.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd6c01c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd640c5ee0 con 0x7fcd6413e080 2023-10-16T08:38:41.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd6c01c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd6413ec00 con 0x7fcd640cc320 2023-10-16T08:38:41.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd6c01c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd640c4900 con 0x7fcd640b2e30 2023-10-16T08:38:41.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd6a5b9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd6413e080 0x7fcd6413e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:41.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd6a5b9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd6413e080 0x7fcd6413e530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46612/0 (socket says 172.21.15.40:46612) 2023-10-16T08:38:41.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd6a5b9640 1 -- 172.21.15.40:0/3081183911 learned_addr learned my addr 172.21.15.40:0/3081183911 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:41.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd695b7640 1 --2- 172.21.15.40:0/3081183911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd640cc320 0x7fcd640c4060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:41.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.560+0000 7fcd69db8640 1 --2- 172.21.15.40:0/3081183911 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd640b2e30 0x7fcd640c59a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:41.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.564+0000 7fcd6a5b9640 1 -- 172.21.15.40:0/3081183911 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd640b2e30 msgr2=0x7fcd640c59a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:41.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.564+0000 7fcd6a5b9640 1 --2- 172.21.15.40:0/3081183911 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd640b2e30 0x7fcd640c59a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:41.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.564+0000 7fcd6a5b9640 1 -- 172.21.15.40:0/3081183911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd640cc320 msgr2=0x7fcd640c4060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:41.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.564+0000 7fcd6a5b9640 1 --2- 172.21.15.40:0/3081183911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd640cc320 0x7fcd640c4060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:41.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.564+0000 7fcd6a5b9640 1 -- 172.21.15.40:0/3081183911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd540094d0 con 0x7fcd6413e080 2023-10-16T08:38:41.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.564+0000 7fcd6a5b9640 1 --2- 172.21.15.40:0/3081183911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd6413e080 0x7fcd6413e530 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c007e30 tx=0x7fcd5c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:41.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.564+0000 7fcd5a7fc640 1 -- 172.21.15.40:0/3081183911 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c014420 con 0x7fcd6413e080 2023-10-16T08:38:41.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.564+0000 7fcd5a7fc640 1 -- 172.21.15.40:0/3081183911 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd5c014580 con 0x7fcd6413e080 2023-10-16T08:38:41.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.564+0000 7fcd5a7fc640 1 -- 172.21.15.40:0/3081183911 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c014930 con 0x7fcd6413e080 2023-10-16T08:38:41.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.564+0000 7fcd6c01c640 1 -- 172.21.15.40:0/3081183911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd640c4be0 con 0x7fcd6413e080 2023-10-16T08:38:41.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.564+0000 7fcd6c01c640 1 -- 172.21.15.40:0/3081183911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd640c51a0 con 0x7fcd6413e080 2023-10-16T08:38:41.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.564+0000 7fcd6c01c640 1 -- 172.21.15.40:0/3081183911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd6404e9c0 con 0x7fcd6413e080 2023-10-16T08:38:41.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.564+0000 7fcd5a7fc640 1 -- 172.21.15.40:0/3081183911 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcd5c014a90 con 0x7fcd6413e080 2023-10-16T08:38:41.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.564+0000 7fcd5a7fc640 1 --2- 172.21.15.40:0/3081183911 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fcd480664d0 0x7fcd48068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:41.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.564+0000 7fcd5a7fc640 1 -- 172.21.15.40:0/3081183911 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fcd5c084450 con 0x7fcd6413e080 2023-10-16T08:38:41.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.564+0000 7fcd69db8640 1 --2- 172.21.15.40:0/3081183911 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fcd480664d0 0x7fcd48068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:41.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.564+0000 7fcd69db8640 1 --2- 172.21.15.40:0/3081183911 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fcd480664d0 0x7fcd48068990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000b930 tx=0x7fcd60008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:41.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.568+0000 7fcd5a7fc640 1 -- 172.21.15.40:0/3081183911 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fcd5c051c20 con 0x7fcd6413e080 2023-10-16T08:38:41.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.764+0000 7fcd6c01c640 1 -- 172.21.15.40:0/3081183911 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcd6415c530 con 0x7fcd6413e080 2023-10-16T08:38:41.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.764+0000 7fcd5a7fc640 1 -- 172.21.15.40:0/3081183911 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3685 (secure 0 0 0) 0x7fcd5c055240 con 0x7fcd6413e080 2023-10-16T08:38:41.768 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:41.768 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/32529848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":32529848},{"type":"v1","addr":"172.21.15.86:6837","nonce":32529848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2044121591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2044121591},{"type":"v1","addr":"172.21.15.86:6836","nonce":2044121591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1275484997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1275484997},{"type":"v1","addr":"172.21.15.40:6833","nonce":1275484997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4287,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6835/148507680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":148507680},{"type":"v1","addr":"172.21.15.40:6835","nonce":148507680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[]} 2023-10-16T08:38:41.769 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 3 2023-10-16T08:38:41.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.764+0000 7fcd6c01c640 1 -- 172.21.15.40:0/3081183911 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fcd480664d0 msgr2=0x7fcd48068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:41.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.764+0000 7fcd6c01c640 1 --2- 172.21.15.40:0/3081183911 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fcd480664d0 0x7fcd48068990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fcd6000b930 tx=0x7fcd60008040 comp rx=0 tx=0).stop 2023-10-16T08:38:41.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.764+0000 7fcd6c01c640 1 -- 172.21.15.40:0/3081183911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd6413e080 msgr2=0x7fcd6413e530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:41.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.764+0000 7fcd6c01c640 1 --2- 172.21.15.40:0/3081183911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd6413e080 0x7fcd6413e530 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c007e30 tx=0x7fcd5c00e4b0 comp rx=0 tx=0).stop 2023-10-16T08:38:41.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.768+0000 7fcd6c01c640 1 -- 172.21.15.40:0/3081183911 shutdown_connections 2023-10-16T08:38:41.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.768+0000 7fcd6c01c640 1 --2- 172.21.15.40:0/3081183911 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fcd480664d0 0x7fcd48068990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:41.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.768+0000 7fcd6c01c640 1 --2- 172.21.15.40:0/3081183911 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcd6413e080 0x7fcd6413e530 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:41.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.768+0000 7fcd6c01c640 1 --2- 172.21.15.40:0/3081183911 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd640cc320 0x7fcd640c4060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:41.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.768+0000 7fcd6c01c640 1 --2- 172.21.15.40:0/3081183911 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcd640b2e30 0x7fcd640c59a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:41.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.768+0000 7fcd6c01c640 1 -- 172.21.15.40:0/3081183911 >> 172.21.15.40:0/3081183911 conn(0x7fcd640b0be0 msgr2=0x7fcd64071590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:41.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.768+0000 7fcd6c01c640 1 -- 172.21.15.40:0/3081183911 shutdown_connections 2023-10-16T08:38:41.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.768+0000 7fcd6c01c640 1 -- 172.21.15.40:0/3081183911 wait complete. 2023-10-16T08:38:41.787 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:38:41.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.900+0000 7f5707608640 1 Processor -- start 2023-10-16T08:38:41.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.900+0000 7f5707608640 1 -- start start 2023-10-16T08:38:41.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.904+0000 7f5707608640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57000ca2c0 0x7f57000ca720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:41.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.904+0000 7f5707608640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f57000b1360 0x7f57000b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:41.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.904+0000 7f5707608640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5700159fc0 con 0x7f5700159be0 2023-10-16T08:38:41.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.904+0000 7f5707608640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57000b1d30 con 0x7f57000b1360 2023-10-16T08:38:41.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.904+0000 7f5707608640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f570006c450 con 0x7f57000ca2c0 2023-10-16T08:38:41.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.904+0000 7f5705ba5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f57000b1360 0x7f57000b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:41.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.904+0000 7f5705ba5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f57000b1360 0x7f57000b17f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46622/0 (socket says 172.21.15.40:46622) 2023-10-16T08:38:41.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.904+0000 7f5705ba5640 1 -- 172.21.15.40:0/2469439445 learned_addr learned my addr 172.21.15.40:0/2469439445 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:41.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.904+0000 7f5704ba3640 1 --2- 172.21.15.40:0/2469439445 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57000ca2c0 0x7f57000ca720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:41.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.904+0000 7f5705ba5640 1 -- 172.21.15.40:0/2469439445 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57000ca2c0 msgr2=0x7f57000ca720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:41.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.904+0000 7f5705ba5640 1 --2- 172.21.15.40:0/2469439445 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57000ca2c0 0x7f57000ca720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:41.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.904+0000 7f5705ba5640 1 -- 172.21.15.40:0/2469439445 >> v1:172.21.15.86:6789/0 conn(0x7f5700159be0 legacy=0x7f57000cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:41.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.904+0000 7f5705ba5640 1 -- 172.21.15.40:0/2469439445 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f570006c730 con 0x7f57000b1360 2023-10-16T08:38:41.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.904+0000 7f5705ba5640 1 --2- 172.21.15.40:0/2469439445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57000b1360 0x7f57000b17f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f56f8009870 tx=0x7f56f801e650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901ebcaf6372b89b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:41.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.904+0000 7f56f3fff640 1 -- 172.21.15.40:0/2469439445 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f80aa640 con 0x7f57000b1360 2023-10-16T08:38:41.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.904+0000 7f56f3fff640 1 -- 172.21.15.40:0/2469439445 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f80aa7a0 con 0x7f57000b1360 2023-10-16T08:38:41.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.904+0000 7f56f3fff640 1 -- 172.21.15.40:0/2469439445 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f80aab30 con 0x7f57000b1360 2023-10-16T08:38:41.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5707608640 1 -- 172.21.15.40:0/2469439445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57000b1360 msgr2=0x7f57000b17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:41.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5707608640 1 --2- 172.21.15.40:0/2469439445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57000b1360 0x7f57000b17f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f56f8009870 tx=0x7f56f801e650 comp rx=0 tx=0).stop 2023-10-16T08:38:41.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5707608640 1 -- 172.21.15.40:0/2469439445 shutdown_connections 2023-10-16T08:38:41.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5707608640 1 --2- 172.21.15.40:0/2469439445 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57000b1360 0x7f57000b17f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:41.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5707608640 1 --2- 172.21.15.40:0/2469439445 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57000ca2c0 0x7f57000ca720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:41.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5707608640 1 -- 172.21.15.40:0/2469439445 >> 172.21.15.40:0/2469439445 conn(0x7f57000cc360 msgr2=0x7f57000cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:41.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5707608640 1 -- 172.21.15.40:0/2469439445 shutdown_connections 2023-10-16T08:38:41.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5707608640 1 -- 172.21.15.40:0/2469439445 wait complete. 2023-10-16T08:38:41.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5707608640 1 Processor -- start 2023-10-16T08:38:41.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5707608640 1 -- start start 2023-10-16T08:38:41.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5707608640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57000b1360 0x7f570014ecd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:41.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5707608640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57000ca2c0 0x7f57000b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:41.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5707608640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5700159be0 0x7f57000c33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:41.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5707608640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57000c9260 con 0x7f57000ca2c0 2023-10-16T08:38:41.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5707608640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57000c85e0 con 0x7f57000b1360 2023-10-16T08:38:41.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5707608640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57000c88c0 con 0x7f5700159be0 2023-10-16T08:38:41.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5704ba3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57000ca2c0 0x7f57000b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:41.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5704ba3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57000ca2c0 0x7f57000b4f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46634/0 (socket says 172.21.15.40:46634) 2023-10-16T08:38:41.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5704ba3640 1 -- 172.21.15.40:0/2314518868 learned_addr learned my addr 172.21.15.40:0/2314518868 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:41.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f57053a4640 1 --2- 172.21.15.40:0/2314518868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57000b1360 0x7f570014ecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:41.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5705ba5640 1 --2- 172.21.15.40:0/2314518868 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5700159be0 0x7f57000c33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:41.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5704ba3640 1 -- 172.21.15.40:0/2314518868 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5700159be0 msgr2=0x7f57000c33b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:41.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5704ba3640 1 --2- 172.21.15.40:0/2314518868 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5700159be0 0x7f57000c33b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:41.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5704ba3640 1 -- 172.21.15.40:0/2314518868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57000b1360 msgr2=0x7f570014ecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:41.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5704ba3640 1 --2- 172.21.15.40:0/2314518868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57000b1360 0x7f570014ecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:41.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5704ba3640 1 -- 172.21.15.40:0/2314518868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56f80094d0 con 0x7f57000ca2c0 2023-10-16T08:38:41.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5704ba3640 1 --2- 172.21.15.40:0/2314518868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57000ca2c0 0x7f57000b4f60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f56f4007e40 tx=0x7f56f400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:41.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f56f1ffb640 1 -- 172.21.15.40:0/2314518868 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f4014620 con 0x7f57000ca2c0 2023-10-16T08:38:41.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f56f1ffb640 1 -- 172.21.15.40:0/2314518868 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56f400ee90 con 0x7f57000ca2c0 2023-10-16T08:38:41.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5707608640 1 -- 172.21.15.40:0/2314518868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57000c8ba0 con 0x7f57000ca2c0 2023-10-16T08:38:41.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f56f1ffb640 1 -- 172.21.15.40:0/2314518868 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f40149c0 con 0x7f57000ca2c0 2023-10-16T08:38:41.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.908+0000 7f5707608640 1 -- 172.21.15.40:0/2314518868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57000b54a0 con 0x7f57000ca2c0 2023-10-16T08:38:41.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.912+0000 7f5707608640 1 -- 172.21.15.40:0/2314518868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f570015e4e0 con 0x7f57000ca2c0 2023-10-16T08:38:41.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.912+0000 7f56f1ffb640 1 -- 172.21.15.40:0/2314518868 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f56f4014b20 con 0x7f57000ca2c0 2023-10-16T08:38:41.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.912+0000 7f56f1ffb640 1 --2- 172.21.15.40:0/2314518868 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f56e40664d0 0x7f56e4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:41.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.912+0000 7f56f1ffb640 1 -- 172.21.15.40:0/2314518868 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f56f40843f0 con 0x7f57000ca2c0 2023-10-16T08:38:41.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.912+0000 7f57053a4640 1 --2- 172.21.15.40:0/2314518868 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f56e40664d0 0x7f56e4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:41.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.912+0000 7f57053a4640 1 --2- 172.21.15.40:0/2314518868 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f56e40664d0 0x7f56e4068990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5700152590 tx=0x7f56fc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:41.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:41.916+0000 7f56f1ffb640 1 -- 172.21.15.40:0/2314518868 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f56f4051bd0 con 0x7f57000ca2c0 2023-10-16T08:38:42.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.068+0000 7f5707608640 1 -- 172.21.15.40:0/2314518868 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f570004e9c0 con 0x7f57000ca2c0 2023-10-16T08:38:42.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.072+0000 7f56f1ffb640 1 -- 172.21.15.40:0/2314518868 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12790 (secure 0 0 0) 0x7f56f40551f0 con 0x7f57000ca2c0 2023-10-16T08:38:42.075 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:42.075 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":16,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:35.290442+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:38:42.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.072+0000 7f5707608640 1 -- 172.21.15.40:0/2314518868 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f56e40664d0 msgr2=0x7f56e4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:42.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.072+0000 7f5707608640 1 --2- 172.21.15.40:0/2314518868 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f56e40664d0 0x7f56e4068990 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f5700152590 tx=0x7f56fc009040 comp rx=0 tx=0).stop 2023-10-16T08:38:42.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.072+0000 7f5707608640 1 -- 172.21.15.40:0/2314518868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57000ca2c0 msgr2=0x7f57000b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:42.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.072+0000 7f5707608640 1 --2- 172.21.15.40:0/2314518868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57000ca2c0 0x7f57000b4f60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f56f4007e40 tx=0x7f56f400e4b0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.072+0000 7f5707608640 1 -- 172.21.15.40:0/2314518868 shutdown_connections 2023-10-16T08:38:42.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.072+0000 7f5707608640 1 --2- 172.21.15.40:0/2314518868 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5700159be0 0x7f57000c33b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.072+0000 7f5707608640 1 --2- 172.21.15.40:0/2314518868 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f56e40664d0 0x7f56e4068990 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.072+0000 7f5707608640 1 --2- 172.21.15.40:0/2314518868 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57000ca2c0 0x7f57000b4f60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.072+0000 7f5707608640 1 --2- 172.21.15.40:0/2314518868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57000b1360 0x7f570014ecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.072+0000 7f5707608640 1 -- 172.21.15.40:0/2314518868 >> 172.21.15.40:0/2314518868 conn(0x7f57000cc360 msgr2=0x7f57000cacc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:42.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.072+0000 7f5707608640 1 -- 172.21.15.40:0/2314518868 shutdown_connections 2023-10-16T08:38:42.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.072+0000 7f5707608640 1 -- 172.21.15.40:0/2314518868 wait complete. 2023-10-16T08:38:42.092 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:38:33.014668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:38:42.092 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:38:42.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.216+0000 7f11f6ab6640 1 Processor -- start 2023-10-16T08:38:42.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.216+0000 7f11f6ab6640 1 -- start start 2023-10-16T08:38:42.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.216+0000 7f11f6ab6640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f11f0159dd0 0x7f11f00c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:42.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.216+0000 7f11f6ab6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f11f00adec0 0x7f11f00c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:42.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.216+0000 7f11f6ab6640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11f015b490 con 0x7f11f006b800 2023-10-16T08:38:42.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.216+0000 7f11f6ab6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f006cd70 con 0x7f11f00adec0 2023-10-16T08:38:42.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.216+0000 7f11f6ab6640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f11f006d230 con 0x7f11f0159dd0 2023-10-16T08:38:42.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.216+0000 7f11effff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f11f00adec0 0x7f11f00c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:42.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.216+0000 7f11effff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f11f00adec0 0x7f11f00c97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46648/0 (socket says 172.21.15.40:46648) 2023-10-16T08:38:42.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.216+0000 7f11effff640 1 -- 172.21.15.40:0/1698804069 learned_addr learned my addr 172.21.15.40:0/1698804069 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:42.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.216+0000 7f11f4852640 1 --2- 172.21.15.40:0/1698804069 >> v2:172.21.15.86:3300/0 conn(0x7f11f0159dd0 0x7f11f00c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:42.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.216+0000 7f11effff640 1 -- 172.21.15.40:0/1698804069 >> v2:172.21.15.86:3300/0 conn(0x7f11f0159dd0 msgr2=0x7f11f00c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:42.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.216+0000 7f11effff640 1 --2- 172.21.15.40:0/1698804069 >> v2:172.21.15.86:3300/0 conn(0x7f11f0159dd0 0x7f11f00c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.216+0000 7f11effff640 1 -- 172.21.15.40:0/1698804069 >> v1:172.21.15.86:6789/0 conn(0x7f11f006b800 legacy=0x7f11f00cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:42.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.216+0000 7f11effff640 1 -- 172.21.15.40:0/1698804069 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11f006cf50 con 0x7f11f00adec0 2023-10-16T08:38:42.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11effff640 1 --2- 172.21.15.40:0/1698804069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11f00adec0 0x7f11f00c97f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f11f006d880 tx=0x7f11e401c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ebdaf9bb3cab76 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:42.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11ef7fe640 1 -- 172.21.15.40:0/1698804069 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e40ac630 con 0x7f11f00adec0 2023-10-16T08:38:42.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11ef7fe640 1 -- 172.21.15.40:0/1698804069 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e40ac790 con 0x7f11f00adec0 2023-10-16T08:38:42.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11ef7fe640 1 -- 172.21.15.40:0/1698804069 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e40acb20 con 0x7f11f00adec0 2023-10-16T08:38:42.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11f6ab6640 1 -- 172.21.15.40:0/1698804069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11f00adec0 msgr2=0x7f11f00c97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:42.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11f6ab6640 1 --2- 172.21.15.40:0/1698804069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11f00adec0 0x7f11f00c97f0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f11f006d880 tx=0x7f11e401c4c0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11f6ab6640 1 -- 172.21.15.40:0/1698804069 shutdown_connections 2023-10-16T08:38:42.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11f6ab6640 1 --2- 172.21.15.40:0/1698804069 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11f00adec0 0x7f11f00c97f0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11f6ab6640 1 --2- 172.21.15.40:0/1698804069 >> v2:172.21.15.86:3300/0 conn(0x7f11f0159dd0 0x7f11f00c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11f6ab6640 1 -- 172.21.15.40:0/1698804069 >> 172.21.15.40:0/1698804069 conn(0x7f11f00ce890 msgr2=0x7f11f00ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:42.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11f6ab6640 1 -- 172.21.15.40:0/1698804069 shutdown_connections 2023-10-16T08:38:42.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11f6ab6640 1 -- 172.21.15.40:0/1698804069 wait complete. 2023-10-16T08:38:42.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11f6ab6640 1 Processor -- start 2023-10-16T08:38:42.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11f6ab6640 1 -- start start 2023-10-16T08:38:42.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11f6ab6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f006b800 0x7f11f01f4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:42.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11f6ab6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11f00adec0 0x7f11f014ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:42.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11f6ab6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11f0159dd0 0x7f11f0152310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:42.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11f6ab6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f0150490 con 0x7f11f00adec0 2023-10-16T08:38:42.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11f6ab6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11f01f4b10 con 0x7f11f006b800 2023-10-16T08:38:42.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11f6ab6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11f01f4df0 con 0x7f11f0159dd0 2023-10-16T08:38:42.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11effff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11f00adec0 0x7f11f014ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:42.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11effff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11f00adec0 0x7f11f014ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46660/0 (socket says 172.21.15.40:46660) 2023-10-16T08:38:42.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.220+0000 7f11effff640 1 -- 172.21.15.40:0/3465482850 learned_addr learned my addr 172.21.15.40:0/3465482850 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:42.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11f4852640 1 --2- 172.21.15.40:0/3465482850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f006b800 0x7f11f01f4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:42.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11f5053640 1 --2- 172.21.15.40:0/3465482850 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11f0159dd0 0x7f11f0152310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:42.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11effff640 1 -- 172.21.15.40:0/3465482850 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11f0159dd0 msgr2=0x7f11f0152310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:42.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11effff640 1 --2- 172.21.15.40:0/3465482850 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11f0159dd0 0x7f11f0152310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11effff640 1 -- 172.21.15.40:0/3465482850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f006b800 msgr2=0x7f11f01f4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:42.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11effff640 1 --2- 172.21.15.40:0/3465482850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f006b800 0x7f11f01f4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11effff640 1 -- 172.21.15.40:0/3465482850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11e40094d0 con 0x7f11f00adec0 2023-10-16T08:38:42.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11f4852640 1 --2- 172.21.15.40:0/3465482850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f006b800 0x7f11f01f4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:42.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11f5053640 1 --2- 172.21.15.40:0/3465482850 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11f0159dd0 0x7f11f0152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:42.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11effff640 1 --2- 172.21.15.40:0/3465482850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11f00adec0 0x7f11f014ff50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f11e40af040 tx=0x7f11e4019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:42.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11ed7fa640 1 -- 172.21.15.40:0/3465482850 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e40b7670 con 0x7f11f00adec0 2023-10-16T08:38:42.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11ed7fa640 1 -- 172.21.15.40:0/3465482850 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11e40b77d0 con 0x7f11f00adec0 2023-10-16T08:38:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11ed7fa640 1 -- 172.21.15.40:0/3465482850 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11e40b7b60 con 0x7f11f00adec0 2023-10-16T08:38:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11f6ab6640 1 -- 172.21.15.40:0/3465482850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11f01f50d0 con 0x7f11f00adec0 2023-10-16T08:38:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11f6ab6640 1 -- 172.21.15.40:0/3465482850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11f01f5690 con 0x7f11f00adec0 2023-10-16T08:38:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11f6ab6640 1 -- 172.21.15.40:0/3465482850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f11f0067a20 con 0x7f11f00adec0 2023-10-16T08:38:42.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11ed7fa640 1 -- 172.21.15.40:0/3465482850 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f11e40b3070 con 0x7f11f00adec0 2023-10-16T08:38:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11ed7fa640 1 --2- 172.21.15.40:0/3465482850 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f11dc0664a0 0x7f11dc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11ed7fa640 1 -- 172.21.15.40:0/3465482850 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f11e411e210 con 0x7f11f00adec0 2023-10-16T08:38:42.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11f4852640 1 --2- 172.21.15.40:0/3465482850 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f11dc0664a0 0x7f11dc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:42.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.224+0000 7f11f4852640 1 --2- 172.21.15.40:0/3465482850 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f11dc0664a0 0x7f11dc068960 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f11e00097a0 tx=0x7f11e0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:42.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.228+0000 7f11ed7fa640 1 -- 172.21.15.40:0/3465482850 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f11e40ebaf0 con 0x7f11f00adec0 2023-10-16T08:38:42.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.388+0000 7f11f6ab6640 1 -- 172.21.15.40:0/3465482850 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f11f00ce690 con 0x7f11f00adec0 2023-10-16T08:38:42.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.392+0000 7f11ed7fa640 1 -- 172.21.15.40:0/3465482850 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v16) v1 ==== 93+0+10 (secure 0 0 0) 0x7f11e40ef110 con 0x7f11f00adec0 2023-10-16T08:38:42.395 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-16T08:38:42.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.392+0000 7f11f6ab6640 1 -- 172.21.15.40:0/3465482850 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f11dc0664a0 msgr2=0x7f11dc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:42.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.392+0000 7f11f6ab6640 1 --2- 172.21.15.40:0/3465482850 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f11dc0664a0 0x7f11dc068960 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f11e00097a0 tx=0x7f11e0008040 comp rx=0 tx=0).stop 2023-10-16T08:38:42.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.392+0000 7f11f6ab6640 1 -- 172.21.15.40:0/3465482850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11f00adec0 msgr2=0x7f11f014ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:42.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.392+0000 7f11f6ab6640 1 --2- 172.21.15.40:0/3465482850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11f00adec0 0x7f11f014ff50 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f11e40af040 tx=0x7f11e4019260 comp rx=0 tx=0).stop 2023-10-16T08:38:42.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.392+0000 7f11f6ab6640 1 -- 172.21.15.40:0/3465482850 shutdown_connections 2023-10-16T08:38:42.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.392+0000 7f11f6ab6640 1 --2- 172.21.15.40:0/3465482850 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11f0159dd0 0x7f11f0152310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.392+0000 7f11f6ab6640 1 --2- 172.21.15.40:0/3465482850 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f11dc0664a0 0x7f11dc068960 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.392+0000 7f11f6ab6640 1 --2- 172.21.15.40:0/3465482850 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f11f00adec0 0x7f11f014ff50 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.392+0000 7f11f6ab6640 1 --2- 172.21.15.40:0/3465482850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f006b800 0x7f11f01f4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.392+0000 7f11f6ab6640 1 -- 172.21.15.40:0/3465482850 >> 172.21.15.40:0/3465482850 conn(0x7f11f00ce890 msgr2=0x7f11f0153c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:42.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.392+0000 7f11f6ab6640 1 -- 172.21.15.40:0/3465482850 shutdown_connections 2023-10-16T08:38:42.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.392+0000 7f11f6ab6640 1 -- 172.21.15.40:0/3465482850 wait complete. 2023-10-16T08:38:42.415 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-16T08:38:42.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.528+0000 7efc36410640 1 Processor -- start 2023-10-16T08:38:42.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.528+0000 7efc36410640 1 -- start start 2023-10-16T08:38:42.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.532+0000 7efc36410640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efc3015b040 0x7efc300ca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:42.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.532+0000 7efc36410640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc300adca0 con 0x7efc300adec0 2023-10-16T08:38:42.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.532+0000 7efc36410640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efc300c5840 con 0x7efc3006c7c0 2023-10-16T08:38:42.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.532+0000 7efc36410640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efc300c9dd0 con 0x7efc3015b040 2023-10-16T08:38:42.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.532+0000 7efc2ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efc3015b040 0x7efc300ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:42.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.532+0000 7efc2ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7efc3015b040 0x7efc300ca740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46662/0 (socket says 172.21.15.40:46662) 2023-10-16T08:38:42.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.532+0000 7efc2f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7efc300adec0 0x7efc300c8fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60618/0 (socket says 172.21.15.40:60618) 2023-10-16T08:38:42.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.532+0000 7efc2ffff640 1 -- 172.21.15.40:0/3554884189 learned_addr learned my addr 172.21.15.40:0/3554884189 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:42.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.532+0000 7efc2ffff640 1 -- 172.21.15.40:0/3554884189 >> v1:172.21.15.86:6789/0 conn(0x7efc3006c7c0 legacy=0x7efc300c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:42.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.532+0000 7efc2ffff640 1 -- 172.21.15.40:0/3554884189 >> v1:172.21.15.40:6789/0 conn(0x7efc300adec0 legacy=0x7efc300c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:42.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.532+0000 7efc2ffff640 1 -- 172.21.15.40:0/3554884189 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc300cf890 con 0x7efc3015b040 2023-10-16T08:38:42.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.532+0000 7efc2ffff640 1 --2- 172.21.15.40:0/3554884189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc3015b040 0x7efc300ca740 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc20009870 tx=0x7efc2001c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5908bd32b5592bab server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:42.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.532+0000 7efc2effd640 1 -- 172.21.15.40:0/3554884189 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc200ab600 con 0x7efc3015b040 2023-10-16T08:38:42.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.532+0000 7efc2effd640 1 -- 172.21.15.40:0/3554884189 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc200a7050 con 0x7efc3015b040 2023-10-16T08:38:42.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.532+0000 7efc2effd640 1 -- 172.21.15.40:0/3554884189 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc200ab980 con 0x7efc3015b040 2023-10-16T08:38:42.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc36410640 1 -- 172.21.15.40:0/3554884189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc3015b040 msgr2=0x7efc300ca740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:42.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc36410640 1 --2- 172.21.15.40:0/3554884189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc3015b040 0x7efc300ca740 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7efc20009870 tx=0x7efc2001c3b0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc36410640 1 -- 172.21.15.40:0/3554884189 shutdown_connections 2023-10-16T08:38:42.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc36410640 1 --2- 172.21.15.40:0/3554884189 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc3015b040 0x7efc300ca740 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc36410640 1 -- 172.21.15.40:0/3554884189 >> 172.21.15.40:0/3554884189 conn(0x7efc300ce960 msgr2=0x7efc300ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:42.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc36410640 1 -- 172.21.15.40:0/3554884189 shutdown_connections 2023-10-16T08:38:42.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc36410640 1 -- 172.21.15.40:0/3554884189 wait complete. 2023-10-16T08:38:42.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc36410640 1 Processor -- start 2023-10-16T08:38:42.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc36410640 1 -- start start 2023-10-16T08:38:42.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc36410640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc3006c7c0 0x7efc301535d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:42.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc36410640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc300adec0 0x7efc30148d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:42.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc36410640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc3015b040 0x7efc3015d200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:42.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc36410640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc30149290 con 0x7efc3006c7c0 2023-10-16T08:38:42.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc36410640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efc3015db70 con 0x7efc3015b040 2023-10-16T08:38:42.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc36410640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efc300c3590 con 0x7efc300adec0 2023-10-16T08:38:42.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc2ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc3006c7c0 0x7efc301535d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:42.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc2ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc3006c7c0 0x7efc301535d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46672/0 (socket says 172.21.15.40:46672) 2023-10-16T08:38:42.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc2ffff640 1 -- 172.21.15.40:0/1944952831 learned_addr learned my addr 172.21.15.40:0/1944952831 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:42.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc349ad640 1 --2- 172.21.15.40:0/1944952831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc3015b040 0x7efc3015d200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:42.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc2f7fe640 1 --2- 172.21.15.40:0/1944952831 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc300adec0 0x7efc30148d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:42.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc2ffff640 1 -- 172.21.15.40:0/1944952831 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc300adec0 msgr2=0x7efc30148d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:42.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc2ffff640 1 --2- 172.21.15.40:0/1944952831 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc300adec0 0x7efc30148d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc2ffff640 1 -- 172.21.15.40:0/1944952831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc3015b040 msgr2=0x7efc3015d200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:42.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc2ffff640 1 --2- 172.21.15.40:0/1944952831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc3015b040 0x7efc3015d200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc2ffff640 1 -- 172.21.15.40:0/1944952831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efc200094d0 con 0x7efc3006c7c0 2023-10-16T08:38:42.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc349ad640 1 --2- 172.21.15.40:0/1944952831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc3015b040 0x7efc3015d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:42.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc2f7fe640 1 --2- 172.21.15.40:0/1944952831 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc300adec0 0x7efc30148d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:42.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc2ffff640 1 --2- 172.21.15.40:0/1944952831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc3006c7c0 0x7efc301535d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efc200ae040 tx=0x7efc2001abf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:42.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc2cff9640 1 -- 172.21.15.40:0/1944952831 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc200b6620 con 0x7efc3006c7c0 2023-10-16T08:38:42.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc2cff9640 1 -- 172.21.15.40:0/1944952831 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efc200b67a0 con 0x7efc3006c7c0 2023-10-16T08:38:42.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc36410640 1 -- 172.21.15.40:0/1944952831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efc300c3870 con 0x7efc3006c7c0 2023-10-16T08:38:42.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc2cff9640 1 -- 172.21.15.40:0/1944952831 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efc200b6b50 con 0x7efc3006c7c0 2023-10-16T08:38:42.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.536+0000 7efc36410640 1 -- 172.21.15.40:0/1944952831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efc3015de50 con 0x7efc3006c7c0 2023-10-16T08:38:42.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.540+0000 7efc36410640 1 -- 172.21.15.40:0/1944952831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efc300c33f0 con 0x7efc3006c7c0 2023-10-16T08:38:42.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.540+0000 7efc2cff9640 1 -- 172.21.15.40:0/1944952831 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7efc200b2040 con 0x7efc3006c7c0 2023-10-16T08:38:42.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.540+0000 7efc2cff9640 1 --2- 172.21.15.40:0/1944952831 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7efc1c0664a0 0x7efc1c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:42.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.540+0000 7efc2cff9640 1 -- 172.21.15.40:0/1944952831 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efc2011dfe0 con 0x7efc3006c7c0 2023-10-16T08:38:42.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.540+0000 7efc2f7fe640 1 --2- 172.21.15.40:0/1944952831 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7efc1c0664a0 0x7efc1c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:42.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.540+0000 7efc2f7fe640 1 --2- 172.21.15.40:0/1944952831 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7efc1c0664a0 0x7efc1c068960 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc24000f70 tx=0x7efc24009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:42.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.544+0000 7efc2cff9640 1 -- 172.21.15.40:0/1944952831 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7efc300c33f0 con 0x7efc3006c7c0 2023-10-16T08:38:42.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.704+0000 7efc36410640 1 -- 172.21.15.40:0/1944952831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7efc300c33f0 con 0x7efc3006c7c0 2023-10-16T08:38:42.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.704+0000 7efc2cff9640 1 -- 172.21.15.40:0/1944952831 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v16) v1 ==== 84+0+65 (secure 0 0 0) 0x7efc200c4060 con 0x7efc3006c7c0 2023-10-16T08:38:42.708 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:42.709 INFO:teuthology.orchestra.run.smithi040.stdout:[ 2023-10-16T08:38:42.709 INFO:teuthology.orchestra.run.smithi040.stdout: { 2023-10-16T08:38:42.709 INFO:teuthology.orchestra.run.smithi040.stdout: "poolnum": 1, 2023-10-16T08:38:42.709 INFO:teuthology.orchestra.run.smithi040.stdout: "poolname": ".mgr" 2023-10-16T08:38:42.709 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-16T08:38:42.710 INFO:teuthology.orchestra.run.smithi040.stdout:] 2023-10-16T08:38:42.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.704+0000 7efc36410640 1 -- 172.21.15.40:0/1944952831 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7efc1c0664a0 msgr2=0x7efc1c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:42.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.704+0000 7efc36410640 1 --2- 172.21.15.40:0/1944952831 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7efc1c0664a0 0x7efc1c068960 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7efc24000f70 tx=0x7efc24009040 comp rx=0 tx=0).stop 2023-10-16T08:38:42.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.704+0000 7efc36410640 1 -- 172.21.15.40:0/1944952831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc3006c7c0 msgr2=0x7efc301535d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:42.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.704+0000 7efc36410640 1 --2- 172.21.15.40:0/1944952831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc3006c7c0 0x7efc301535d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efc200ae040 tx=0x7efc2001abf0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.704+0000 7efc36410640 1 -- 172.21.15.40:0/1944952831 shutdown_connections 2023-10-16T08:38:42.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.704+0000 7efc36410640 1 --2- 172.21.15.40:0/1944952831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc3015b040 0x7efc3015d200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.704+0000 7efc36410640 1 --2- 172.21.15.40:0/1944952831 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7efc300adec0 0x7efc30148d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.704+0000 7efc36410640 1 --2- 172.21.15.40:0/1944952831 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7efc1c0664a0 0x7efc1c068960 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.704+0000 7efc36410640 1 --2- 172.21.15.40:0/1944952831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7efc3006c7c0 0x7efc301535d0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.704+0000 7efc36410640 1 -- 172.21.15.40:0/1944952831 >> 172.21.15.40:0/1944952831 conn(0x7efc300ce960 msgr2=0x7efc300ca400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:42.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.704+0000 7efc36410640 1 -- 172.21.15.40:0/1944952831 shutdown_connections 2023-10-16T08:38:42.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.704+0000 7efc36410640 1 -- 172.21.15.40:0/1944952831 wait complete. 2023-10-16T08:38:42.726 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-16T08:38:42.726 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-16T08:38:42.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.848+0000 7f9ad24c7640 1 Processor -- start 2023-10-16T08:38:42.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.848+0000 7f9ad24c7640 1 -- start start 2023-10-16T08:38:42.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.848+0000 7f9ad24c7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9acc1603b0 0x7f9acc0d0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:42.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.848+0000 7f9ad24c7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9acc0ce010 0x7f9acc0b5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:42.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.848+0000 7f9ad24c7640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9acc160790 con 0x7f9acc0d1160 2023-10-16T08:38:42.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.848+0000 7f9ad24c7640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9acc0d22f0 con 0x7f9acc0ce010 2023-10-16T08:38:42.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.848+0000 7f9ad24c7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc0d1670 con 0x7f9acc1603b0 2023-10-16T08:38:42.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.848+0000 7f9ad0a64640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9acc0ce010 0x7f9acc0b5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:42.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.848+0000 7f9ad0a64640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9acc0ce010 0x7f9acc0b5cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46676/0 (socket says 172.21.15.40:46676) 2023-10-16T08:38:42.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.848+0000 7f9ad0a64640 1 -- 172.21.15.40:0/187867575 learned_addr learned my addr 172.21.15.40:0/187867575 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:42.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.848+0000 7f9ad0a64640 1 -- 172.21.15.40:0/187867575 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9acc1603b0 msgr2=0x7f9acc0d0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:42.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.848+0000 7f9acbfff640 1 --2- 172.21.15.40:0/187867575 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9acc1603b0 0x7f9acc0d0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:42.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.848+0000 7f9ad0a64640 1 --2- 172.21.15.40:0/187867575 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9acc1603b0 0x7f9acc0d0c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.848+0000 7f9ad0a64640 1 -- 172.21.15.40:0/187867575 >> v1:172.21.15.86:6789/0 conn(0x7f9acc0d1160 legacy=0x7f9acc0cd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:42.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.848+0000 7f9ad0a64640 1 -- 172.21.15.40:0/187867575 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc0d1950 con 0x7f9acc0ce010 2023-10-16T08:38:42.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9ad0a64640 1 --2- 172.21.15.40:0/187867575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc0ce010 0x7f9acc0b5cd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0b2500 tx=0x7f9ac401c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ed28607e372a0a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:42.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9acaffd640 1 -- 172.21.15.40:0/187867575 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac40a5770 con 0x7f9acc0ce010 2023-10-16T08:38:42.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9acaffd640 1 -- 172.21.15.40:0/187867575 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac40a58d0 con 0x7f9acc0ce010 2023-10-16T08:38:42.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9acaffd640 1 -- 172.21.15.40:0/187867575 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac40a5c60 con 0x7f9acc0ce010 2023-10-16T08:38:42.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9ad24c7640 1 -- 172.21.15.40:0/187867575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc0ce010 msgr2=0x7f9acc0b5cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:42.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9ad24c7640 1 --2- 172.21.15.40:0/187867575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc0ce010 0x7f9acc0b5cd0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f9acc0b2500 tx=0x7f9ac401c8f0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9ad24c7640 1 -- 172.21.15.40:0/187867575 shutdown_connections 2023-10-16T08:38:42.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9ad24c7640 1 --2- 172.21.15.40:0/187867575 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc0ce010 0x7f9acc0b5cd0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9ad24c7640 1 --2- 172.21.15.40:0/187867575 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9acc1603b0 0x7f9acc0d0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9ad24c7640 1 -- 172.21.15.40:0/187867575 >> 172.21.15.40:0/187867575 conn(0x7f9acc06a290 msgr2=0x7f9acc06a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:42.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9ad24c7640 1 -- 172.21.15.40:0/187867575 shutdown_connections 2023-10-16T08:38:42.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9ad24c7640 1 -- 172.21.15.40:0/187867575 wait complete. 2023-10-16T08:38:42.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9ad24c7640 1 Processor -- start 2023-10-16T08:38:42.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9ad24c7640 1 -- start start 2023-10-16T08:38:42.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9ad24c7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9acc0ce010 0x7f9acc15d040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:42.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9ad24c7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9acc0d1160 0x7f9acc14dd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:42.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9ad24c7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc1603b0 0x7f9acc14c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:42.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9ad24c7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc156700 con 0x7f9acc1603b0 2023-10-16T08:38:42.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9ad24c7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc14c6b0 con 0x7f9acc0d1160 2023-10-16T08:38:42.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9ad24c7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9acc15d620 con 0x7f9acc0ce010 2023-10-16T08:38:42.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9ad0a64640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc1603b0 0x7f9acc14c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:42.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9ad0a64640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc1603b0 0x7f9acc14c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46692/0 (socket says 172.21.15.40:46692) 2023-10-16T08:38:42.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9ad0a64640 1 -- 172.21.15.40:0/2108445298 learned_addr learned my addr 172.21.15.40:0/2108445298 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:42.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9acb7fe640 1 --2- 172.21.15.40:0/2108445298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9acc0d1160 0x7f9acc14dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:42.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.852+0000 7f9acbfff640 1 --2- 172.21.15.40:0/2108445298 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9acc0ce010 0x7f9acc15d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:42.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.856+0000 7f9ad0a64640 1 -- 172.21.15.40:0/2108445298 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9acc0ce010 msgr2=0x7f9acc15d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:42.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.856+0000 7f9ad0a64640 1 --2- 172.21.15.40:0/2108445298 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9acc0ce010 0x7f9acc15d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.856+0000 7f9ad0a64640 1 -- 172.21.15.40:0/2108445298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9acc0d1160 msgr2=0x7f9acc14dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:42.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.856+0000 7f9ad0a64640 1 --2- 172.21.15.40:0/2108445298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9acc0d1160 0x7f9acc14dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:42.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.856+0000 7f9ad0a64640 1 -- 172.21.15.40:0/2108445298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ac4009b30 con 0x7f9acc1603b0 2023-10-16T08:38:42.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.856+0000 7f9ad0a64640 1 --2- 172.21.15.40:0/2108445298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc1603b0 0x7f9acc14c170 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9ac40af040 tx=0x7f9ac4019690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:42.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.856+0000 7f9ac8ff9640 1 -- 172.21.15.40:0/2108445298 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac40b8670 con 0x7f9acc1603b0 2023-10-16T08:38:42.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.856+0000 7f9ac8ff9640 1 -- 172.21.15.40:0/2108445298 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ac40b8810 con 0x7f9acc1603b0 2023-10-16T08:38:42.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.856+0000 7f9ad24c7640 1 -- 172.21.15.40:0/2108445298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9acc15d900 con 0x7f9acc1603b0 2023-10-16T08:38:42.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.856+0000 7f9ac8ff9640 1 -- 172.21.15.40:0/2108445298 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ac40b8bc0 con 0x7f9acc1603b0 2023-10-16T08:38:42.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.856+0000 7f9ad24c7640 1 -- 172.21.15.40:0/2108445298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9acc156320 con 0x7f9acc1603b0 2023-10-16T08:38:42.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.856+0000 7f9ad24c7640 1 -- 172.21.15.40:0/2108445298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9acc156840 con 0x7f9acc1603b0 2023-10-16T08:38:42.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.856+0000 7f9ac8ff9640 1 -- 172.21.15.40:0/2108445298 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9ac40b8d20 con 0x7f9acc1603b0 2023-10-16T08:38:42.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.856+0000 7f9ac8ff9640 1 --2- 172.21.15.40:0/2108445298 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f9ab80664a0 0x7f9ab8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:42.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.856+0000 7f9ac8ff9640 1 -- 172.21.15.40:0/2108445298 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9ac411eff0 con 0x7f9acc1603b0 2023-10-16T08:38:42.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.856+0000 7f9acbfff640 1 --2- 172.21.15.40:0/2108445298 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f9ab80664a0 0x7f9ab8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:42.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.856+0000 7f9acbfff640 1 --2- 172.21.15.40:0/2108445298 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f9ab80664a0 0x7f9ab8068960 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9acc15cfd0 tx=0x7f9abc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:42.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:42.860+0000 7f9ac8ff9640 1 -- 172.21.15.40:0/2108445298 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f9acc156840 con 0x7f9acc1603b0 2023-10-16T08:38:43.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.012+0000 7f9ad24c7640 1 -- 172.21.15.40:0/2108445298 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f9acc160790 con 0x7f9acc1603b0 2023-10-16T08:38:43.190 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:38:43.194 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:38:43.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.588+0000 7f9ac8ff9640 1 -- 172.21.15.40:0/2108445298 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f9acc160790 con 0x7f9acc1603b0 2023-10-16T08:38:43.595 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' created 2023-10-16T08:38:43.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.592+0000 7f9ad24c7640 1 -- 172.21.15.40:0/2108445298 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f9ab80664a0 msgr2=0x7f9ab8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:43.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.592+0000 7f9ad24c7640 1 --2- 172.21.15.40:0/2108445298 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f9ab80664a0 0x7f9ab8068960 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9acc15cfd0 tx=0x7f9abc008040 comp rx=0 tx=0).stop 2023-10-16T08:38:43.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.592+0000 7f9ad24c7640 1 -- 172.21.15.40:0/2108445298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc1603b0 msgr2=0x7f9acc14c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:43.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.592+0000 7f9ad24c7640 1 --2- 172.21.15.40:0/2108445298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc1603b0 0x7f9acc14c170 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f9ac40af040 tx=0x7f9ac4019690 comp rx=0 tx=0).stop 2023-10-16T08:38:43.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.592+0000 7f9ad24c7640 1 -- 172.21.15.40:0/2108445298 shutdown_connections 2023-10-16T08:38:43.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.592+0000 7f9ad24c7640 1 --2- 172.21.15.40:0/2108445298 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f9ab80664a0 0x7f9ab8068960 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:43.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.592+0000 7f9ad24c7640 1 --2- 172.21.15.40:0/2108445298 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9acc1603b0 0x7f9acc14c170 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:43.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.592+0000 7f9ad24c7640 1 --2- 172.21.15.40:0/2108445298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9acc0d1160 0x7f9acc14dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:43.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.592+0000 7f9ad24c7640 1 --2- 172.21.15.40:0/2108445298 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9acc0ce010 0x7f9acc15d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:43.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.592+0000 7f9ad24c7640 1 -- 172.21.15.40:0/2108445298 >> 172.21.15.40:0/2108445298 conn(0x7f9acc06a290 msgr2=0x7f9acc0c9c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:43.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.596+0000 7f9ad24c7640 1 -- 172.21.15.40:0/2108445298 shutdown_connections 2023-10-16T08:38:43.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.596+0000 7f9ad24c7640 1 -- 172.21.15.40:0/2108445298 wait complete. 2023-10-16T08:38:43.625 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-16T08:38:43.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.756+0000 7f7746c49640 1 Processor -- start 2023-10-16T08:38:43.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.756+0000 7f7746c49640 1 -- start start 2023-10-16T08:38:43.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.756+0000 7f7746c49640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f774015ae20 0x7f774006cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:43.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.756+0000 7f7746c49640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f77400ca040 0x7f77400b1590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:43.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.756+0000 7f7746c49640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f77400b1ad0 0x7f77400b01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:43.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.756+0000 7f7746c49640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f774006d200 con 0x7f77400b1ad0 2023-10-16T08:38:43.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.756+0000 7f7746c49640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f774006f5a0 con 0x7f77400ca040 2023-10-16T08:38:43.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.756+0000 7f7746c49640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774006b3c0 con 0x7f774015ae20 2023-10-16T08:38:43.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.756+0000 7f77451e6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f77400b1ad0 0x7f77400b01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:43.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.756+0000 7f77451e6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f77400b1ad0 0x7f77400b01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46698/0 (socket says 172.21.15.40:46698) 2023-10-16T08:38:43.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.756+0000 7f77451e6640 1 -- 172.21.15.40:0/2736993484 learned_addr learned my addr 172.21.15.40:0/2736993484 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:43.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.756+0000 7f77449e5640 1 --2- 172.21.15.40:0/2736993484 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f774015ae20 0x7f774006cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:43.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.756+0000 7f773ffff640 1 --2- 172.21.15.40:0/2736993484 >> v2:172.21.15.86:3300/0 conn(0x7f77400ca040 0x7f77400b1590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:43.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.756+0000 7f77451e6640 1 -- 172.21.15.40:0/2736993484 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f774015ae20 msgr2=0x7f774006cb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:43.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.756+0000 7f77451e6640 1 --2- 172.21.15.40:0/2736993484 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f774015ae20 0x7f774006cb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:43.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.756+0000 7f77451e6640 1 -- 172.21.15.40:0/2736993484 >> v2:172.21.15.86:3300/0 conn(0x7f77400ca040 msgr2=0x7f77400b1590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:43.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.756+0000 7f77451e6640 1 --2- 172.21.15.40:0/2736993484 >> v2:172.21.15.86:3300/0 conn(0x7f77400ca040 0x7f77400b1590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:43.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.756+0000 7f77451e6640 1 -- 172.21.15.40:0/2736993484 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f77400ca8e0 con 0x7f77400b1ad0 2023-10-16T08:38:43.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.756+0000 7f773ffff640 1 --2- 172.21.15.40:0/2736993484 >> v2:172.21.15.86:3300/0 conn(0x7f77400ca040 0x7f77400b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:38:43.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.760+0000 7f77451e6640 1 --2- 172.21.15.40:0/2736993484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77400b1ad0 0x7f77400b01c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f77400ce630 tx=0x7f773801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e73f2c722c1aba7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:43.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.760+0000 7f773f7fe640 1 -- 172.21.15.40:0/2736993484 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77380a56d0 con 0x7f77400b1ad0 2023-10-16T08:38:43.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.760+0000 7f773f7fe640 1 -- 172.21.15.40:0/2736993484 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77380a5830 con 0x7f77400b1ad0 2023-10-16T08:38:43.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.760+0000 7f773f7fe640 1 -- 172.21.15.40:0/2736993484 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77380a5bc0 con 0x7f77400b1ad0 2023-10-16T08:38:43.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.760+0000 7f7746c49640 1 -- 172.21.15.40:0/2736993484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77400b1ad0 msgr2=0x7f77400b01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:43.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.760+0000 7f7746c49640 1 --2- 172.21.15.40:0/2736993484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77400b1ad0 0x7f77400b01c0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f77400ce630 tx=0x7f773801c940 comp rx=0 tx=0).stop 2023-10-16T08:38:43.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.760+0000 7f7746c49640 1 -- 172.21.15.40:0/2736993484 shutdown_connections 2023-10-16T08:38:43.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.760+0000 7f7746c49640 1 --2- 172.21.15.40:0/2736993484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77400b1ad0 0x7f77400b01c0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:43.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.760+0000 7f7746c49640 1 --2- 172.21.15.40:0/2736993484 >> v2:172.21.15.86:3300/0 conn(0x7f77400ca040 0x7f77400b1590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:43.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.760+0000 7f7746c49640 1 --2- 172.21.15.40:0/2736993484 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f774015ae20 0x7f774006cb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:43.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.760+0000 7f7746c49640 1 -- 172.21.15.40:0/2736993484 >> 172.21.15.40:0/2736993484 conn(0x7f77400cb1c0 msgr2=0x7f77400cb5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:43.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.760+0000 7f7746c49640 1 -- 172.21.15.40:0/2736993484 shutdown_connections 2023-10-16T08:38:43.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.760+0000 7f7746c49640 1 -- 172.21.15.40:0/2736993484 wait complete. 2023-10-16T08:38:43.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.760+0000 7f7746c49640 1 Processor -- start 2023-10-16T08:38:43.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.760+0000 7f7746c49640 1 -- start start 2023-10-16T08:38:43.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f7746c49640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77400b1ad0 0x7f77400c7440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:43.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f7746c49640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f77400ca040 0x7f7740154bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:43.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f7746c49640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f774015ae20 0x7f77401487e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:43.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f7746c49640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77400c7c90 con 0x7f77400b1ad0 2023-10-16T08:38:43.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f7746c49640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f774015b5b0 con 0x7f774015ae20 2023-10-16T08:38:43.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f7746c49640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f774015b890 con 0x7f77400ca040 2023-10-16T08:38:43.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f77449e5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77400b1ad0 0x7f77400c7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:43.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f77449e5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77400b1ad0 0x7f77400c7440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:46714/0 (socket says 172.21.15.40:46714) 2023-10-16T08:38:43.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f77449e5640 1 -- 172.21.15.40:0/191522812 learned_addr learned my addr 172.21.15.40:0/191522812 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:43.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f77451e6640 1 --2- 172.21.15.40:0/191522812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f774015ae20 0x7f77401487e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:43.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f773ffff640 1 --2- 172.21.15.40:0/191522812 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f77400ca040 0x7f7740154bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:43.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f77449e5640 1 -- 172.21.15.40:0/191522812 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f77400ca040 msgr2=0x7f7740154bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:43.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f77449e5640 1 --2- 172.21.15.40:0/191522812 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f77400ca040 0x7f7740154bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:43.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f77449e5640 1 -- 172.21.15.40:0/191522812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f774015ae20 msgr2=0x7f77401487e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:43.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f77449e5640 1 --2- 172.21.15.40:0/191522812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f774015ae20 0x7f77401487e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:43.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f77449e5640 1 -- 172.21.15.40:0/191522812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7738009b30 con 0x7f77400b1ad0 2023-10-16T08:38:43.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f77449e5640 1 --2- 172.21.15.40:0/191522812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77400b1ad0 0x7f77400c7440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f773000e870 tx=0x7f773000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:43.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f773d7fa640 1 -- 172.21.15.40:0/191522812 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7730009510 con 0x7f77400b1ad0 2023-10-16T08:38:43.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f773d7fa640 1 -- 172.21.15.40:0/191522812 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f773000eec0 con 0x7f77400b1ad0 2023-10-16T08:38:43.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f7746c49640 1 -- 172.21.15.40:0/191522812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f774015b380 con 0x7f77400b1ad0 2023-10-16T08:38:43.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f773d7fa640 1 -- 172.21.15.40:0/191522812 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f773000cb40 con 0x7f77400b1ad0 2023-10-16T08:38:43.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f7746c49640 1 -- 172.21.15.40:0/191522812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f774015be50 con 0x7f77400b1ad0 2023-10-16T08:38:43.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f7746c49640 1 -- 172.21.15.40:0/191522812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f774004f100 con 0x7f77400b1ad0 2023-10-16T08:38:43.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f773d7fa640 1 -- 172.21.15.40:0/191522812 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f773000cca0 con 0x7f77400b1ad0 2023-10-16T08:38:43.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f773d7fa640 1 --2- 172.21.15.40:0/191522812 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f772c0664a0 0x7f772c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:43.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f773d7fa640 1 -- 172.21.15.40:0/191522812 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f77300843c0 con 0x7f77400b1ad0 2023-10-16T08:38:43.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.764+0000 7f773ffff640 1 --2- 172.21.15.40:0/191522812 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f772c0664a0 0x7f772c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:43.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.768+0000 7f773ffff640 1 --2- 172.21.15.40:0/191522812 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f772c0664a0 0x7f772c068960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f77400b3920 tx=0x7f7734008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:43.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.768+0000 7f773d7fa640 1 -- 172.21.15.40:0/191522812 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f7730051b90 con 0x7f77400b1ad0 2023-10-16T08:38:43.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:43.924+0000 7f7746c49640 1 -- 172.21.15.40:0/191522812 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f774004e9c0 con 0x7f77400b1ad0 2023-10-16T08:38:44.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.604+0000 7f773d7fa640 1 -- 172.21.15.40:0/191522812 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v18) v1 ==== 170+0+0 (secure 0 0 0) 0x7f77300551b0 con 0x7f77400b1ad0 2023-10-16T08:38:44.608 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' created 2023-10-16T08:38:44.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.604+0000 7f7746c49640 1 -- 172.21.15.40:0/191522812 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f772c0664a0 msgr2=0x7f772c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:44.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.604+0000 7f7746c49640 1 --2- 172.21.15.40:0/191522812 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f772c0664a0 0x7f772c068960 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f77400b3920 tx=0x7f7734008040 comp rx=0 tx=0).stop 2023-10-16T08:38:44.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.604+0000 7f7746c49640 1 -- 172.21.15.40:0/191522812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77400b1ad0 msgr2=0x7f77400c7440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:44.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.604+0000 7f7746c49640 1 --2- 172.21.15.40:0/191522812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77400b1ad0 0x7f77400c7440 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f773000e870 tx=0x7f773000ec70 comp rx=0 tx=0).stop 2023-10-16T08:38:44.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.604+0000 7f7746c49640 1 -- 172.21.15.40:0/191522812 shutdown_connections 2023-10-16T08:38:44.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.604+0000 7f7746c49640 1 --2- 172.21.15.40:0/191522812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f774015ae20 0x7f77401487e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:44.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.604+0000 7f7746c49640 1 --2- 172.21.15.40:0/191522812 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f77400ca040 0x7f7740154bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:44.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.604+0000 7f7746c49640 1 --2- 172.21.15.40:0/191522812 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f772c0664a0 0x7f772c068960 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:44.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.604+0000 7f7746c49640 1 --2- 172.21.15.40:0/191522812 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f77400b1ad0 0x7f77400c7440 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:44.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.608+0000 7f7746c49640 1 -- 172.21.15.40:0/191522812 >> 172.21.15.40:0/191522812 conn(0x7f77400cb1c0 msgr2=0x7f77400c1990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:44.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.608+0000 7f7746c49640 1 -- 172.21.15.40:0/191522812 shutdown_connections 2023-10-16T08:38:44.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.608+0000 7f7746c49640 1 -- 172.21.15.40:0/191522812 wait complete. 2023-10-16T08:38:44.644 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-16T08:38:44.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.772+0000 7f225a45a640 1 Processor -- start 2023-10-16T08:38:44.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.772+0000 7f225a45a640 1 -- start start 2023-10-16T08:38:44.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.772+0000 7f225a45a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22540afb90 0x7f22540cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:44.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.772+0000 7f225a45a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f22540d00f0 0x7f2254164750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:44.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.772+0000 7f225a45a640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22540cd2e0 con 0x7f22540ce540 2023-10-16T08:38:44.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.772+0000 7f225a45a640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22540cde80 con 0x7f22540d00f0 2023-10-16T08:38:44.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.772+0000 7f225a45a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22540cf930 con 0x7f22540afb90 2023-10-16T08:38:44.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.772+0000 7f22589f7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f22540d00f0 0x7f2254164750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:44.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.772+0000 7f22589f7640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f22540d00f0 0x7f2254164750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57978/0 (socket says 172.21.15.40:57978) 2023-10-16T08:38:44.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.772+0000 7f22589f7640 1 -- 172.21.15.40:0/124693688 learned_addr learned my addr 172.21.15.40:0/124693688 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:44.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.772+0000 7f22537fe640 1 --2- 172.21.15.40:0/124693688 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22540afb90 0x7f22540cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:44.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.772+0000 7f22589f7640 1 -- 172.21.15.40:0/124693688 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22540afb90 msgr2=0x7f22540cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:44.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.772+0000 7f22589f7640 1 --2- 172.21.15.40:0/124693688 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22540afb90 0x7f22540cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:44.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.772+0000 7f22589f7640 1 -- 172.21.15.40:0/124693688 >> v1:172.21.15.86:6789/0 conn(0x7f22540ce540 legacy=0x7f22540cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:44.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.772+0000 7f22589f7640 1 -- 172.21.15.40:0/124693688 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22540cc6d0 con 0x7f22540d00f0 2023-10-16T08:38:44.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.772+0000 7f22537fe640 1 --2- 172.21.15.40:0/124693688 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22540afb90 0x7f22540cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:44.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.772+0000 7f22589f7640 1 --2- 172.21.15.40:0/124693688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22540d00f0 0x7f2254164750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f224c009870 tx=0x7f224c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=303e06f5af7e583f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:44.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f2252ffd640 1 -- 172.21.15.40:0/124693688 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c0aa620 con 0x7f22540d00f0 2023-10-16T08:38:44.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f2252ffd640 1 -- 172.21.15.40:0/124693688 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f224c0aa780 con 0x7f22540d00f0 2023-10-16T08:38:44.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f2252ffd640 1 -- 172.21.15.40:0/124693688 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c0aab10 con 0x7f22540d00f0 2023-10-16T08:38:44.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f225a45a640 1 -- 172.21.15.40:0/124693688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22540d00f0 msgr2=0x7f2254164750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:44.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f225a45a640 1 --2- 172.21.15.40:0/124693688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22540d00f0 0x7f2254164750 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f224c009870 tx=0x7f224c01c500 comp rx=0 tx=0).stop 2023-10-16T08:38:44.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f225a45a640 1 -- 172.21.15.40:0/124693688 shutdown_connections 2023-10-16T08:38:44.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f225a45a640 1 --2- 172.21.15.40:0/124693688 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22540d00f0 0x7f2254164750 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:44.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f225a45a640 1 --2- 172.21.15.40:0/124693688 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22540afb90 0x7f22540cfbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:44.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f225a45a640 1 -- 172.21.15.40:0/124693688 >> 172.21.15.40:0/124693688 conn(0x7f2254065e80 msgr2=0x7f2254066290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:44.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f225a45a640 1 -- 172.21.15.40:0/124693688 shutdown_connections 2023-10-16T08:38:44.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f225a45a640 1 -- 172.21.15.40:0/124693688 wait complete. 2023-10-16T08:38:44.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f225a45a640 1 Processor -- start 2023-10-16T08:38:44.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f225a45a640 1 -- start start 2023-10-16T08:38:44.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f225a45a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22540afb90 0x7f22540b4370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:44.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f225a45a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22540ce540 0x7f22540b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:44.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f225a45a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22540d00f0 0x7f2254162aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:44.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f225a45a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225415cb20 con 0x7f22540d00f0 2023-10-16T08:38:44.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f225a45a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2254164af0 con 0x7f22540ce540 2023-10-16T08:38:44.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f225a45a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225414e720 con 0x7f22540afb90 2023-10-16T08:38:44.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f22589f7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22540d00f0 0x7f2254162aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:44.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f22589f7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22540d00f0 0x7f2254162aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:57984/0 (socket says 172.21.15.40:57984) 2023-10-16T08:38:44.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f22589f7640 1 -- 172.21.15.40:0/1316322623 learned_addr learned my addr 172.21.15.40:0/1316322623 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:44.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f2253fff640 1 --2- 172.21.15.40:0/1316322623 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22540afb90 0x7f22540b4370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:44.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f22589f7640 1 -- 172.21.15.40:0/1316322623 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22540afb90 msgr2=0x7f22540b4370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:44.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f22589f7640 1 --2- 172.21.15.40:0/1316322623 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22540afb90 0x7f22540b4370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:44.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f22537fe640 1 --2- 172.21.15.40:0/1316322623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22540ce540 0x7f22540b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:44.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f22589f7640 1 -- 172.21.15.40:0/1316322623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22540ce540 msgr2=0x7f22540b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:44.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f22589f7640 1 --2- 172.21.15.40:0/1316322623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22540ce540 0x7f22540b48b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:44.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f22589f7640 1 -- 172.21.15.40:0/1316322623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f224c0094d0 con 0x7f22540d00f0 2023-10-16T08:38:44.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f2253fff640 1 --2- 172.21.15.40:0/1316322623 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22540afb90 0x7f22540b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:44.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f22589f7640 1 --2- 172.21.15.40:0/1316322623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22540d00f0 0x7f2254162aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f224c0ac040 tx=0x7f224c01ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:44.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f2250ff9640 1 -- 172.21.15.40:0/1316322623 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c0bd5f0 con 0x7f22540d00f0 2023-10-16T08:38:44.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f2250ff9640 1 -- 172.21.15.40:0/1316322623 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f224c0bd770 con 0x7f22540d00f0 2023-10-16T08:38:44.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f225a45a640 1 -- 172.21.15.40:0/1316322623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f225415d2f0 con 0x7f22540d00f0 2023-10-16T08:38:44.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f2250ff9640 1 -- 172.21.15.40:0/1316322623 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c0bdb20 con 0x7f22540d00f0 2023-10-16T08:38:44.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.776+0000 7f225a45a640 1 -- 172.21.15.40:0/1316322623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f225415d870 con 0x7f22540d00f0 2023-10-16T08:38:44.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.780+0000 7f225a45a640 1 -- 172.21.15.40:0/1316322623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f225404e9c0 con 0x7f22540d00f0 2023-10-16T08:38:44.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.780+0000 7f2250ff9640 1 -- 172.21.15.40:0/1316322623 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f224c0b33f0 con 0x7f22540d00f0 2023-10-16T08:38:44.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.780+0000 7f2250ff9640 1 --2- 172.21.15.40:0/1316322623 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f22380664a0 0x7f2238068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:44.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.780+0000 7f2250ff9640 1 -- 172.21.15.40:0/1316322623 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f224c11e7d0 con 0x7f22540d00f0 2023-10-16T08:38:44.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.780+0000 7f2253fff640 1 --2- 172.21.15.40:0/1316322623 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f22380664a0 0x7f2238068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:44.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.780+0000 7f2253fff640 1 --2- 172.21.15.40:0/1316322623 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f22380664a0 0x7f2238068960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2244007e90 tx=0x7f22440073a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:44.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.784+0000 7f2250ff9640 1 -- 172.21.15.40:0/1316322623 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f224c0ebe80 con 0x7f22540d00f0 2023-10-16T08:38:44.983 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:44.980+0000 7f225a45a640 1 -- 172.21.15.40:0/1316322623 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f22540b5750 con 0x7f22540d00f0 2023-10-16T08:38:45.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:45.924+0000 7f2250ff9640 1 -- 172.21.15.40:0/1316322623 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f224c0c30c0 con 0x7f22540d00f0 2023-10-16T08:38:45.929 INFO:teuthology.orchestra.run.smithi040.stderr:new fs with metadata pool 2 and data pool 3 2023-10-16T08:38:45.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:45.928+0000 7f225a45a640 1 -- 172.21.15.40:0/1316322623 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f22380664a0 msgr2=0x7f2238068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:45.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:45.928+0000 7f225a45a640 1 --2- 172.21.15.40:0/1316322623 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f22380664a0 0x7f2238068960 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f2244007e90 tx=0x7f22440073a0 comp rx=0 tx=0).stop 2023-10-16T08:38:45.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:45.928+0000 7f225a45a640 1 -- 172.21.15.40:0/1316322623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22540d00f0 msgr2=0x7f2254162aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:45.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:45.928+0000 7f225a45a640 1 --2- 172.21.15.40:0/1316322623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22540d00f0 0x7f2254162aa0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f224c0ac040 tx=0x7f224c01ad40 comp rx=0 tx=0).stop 2023-10-16T08:38:45.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:45.928+0000 7f225a45a640 1 -- 172.21.15.40:0/1316322623 shutdown_connections 2023-10-16T08:38:45.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:45.928+0000 7f225a45a640 1 --2- 172.21.15.40:0/1316322623 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f22380664a0 0x7f2238068960 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:45.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:45.928+0000 7f225a45a640 1 --2- 172.21.15.40:0/1316322623 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f22540d00f0 0x7f2254162aa0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:45.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:45.928+0000 7f225a45a640 1 --2- 172.21.15.40:0/1316322623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22540ce540 0x7f22540b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:45.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:45.928+0000 7f225a45a640 1 --2- 172.21.15.40:0/1316322623 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f22540afb90 0x7f22540b4370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:45.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:45.928+0000 7f225a45a640 1 -- 172.21.15.40:0/1316322623 >> 172.21.15.40:0/1316322623 conn(0x7f2254065e80 msgr2=0x7f22540cd770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:45.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:45.928+0000 7f225a45a640 1 -- 172.21.15.40:0/1316322623 shutdown_connections 2023-10-16T08:38:45.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:45.928+0000 7f225a45a640 1 -- 172.21.15.40:0/1316322623 wait complete. 2023-10-16T08:38:45.959 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:38:46.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8e06c5640 1 Processor -- start 2023-10-16T08:38:46.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8e06c5640 1 -- start start 2023-10-16T08:38:46.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8e06c5640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fc8d815d180 0x7fc8d806b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:46.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8e06c5640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d80cf480 con 0x7fc8d80ad820 2023-10-16T08:38:46.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8e06c5640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d80cf760 con 0x7fc8d815d560 2023-10-16T08:38:46.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8e06c5640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d80c8fb0 con 0x7fc8d815d180 2023-10-16T08:38:46.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8dec62640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc8d80ad820 0x7fc8d80cd220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58260/0 (socket says 172.21.15.40:58260) 2023-10-16T08:38:46.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8dec62640 1 -- 172.21.15.40:0/397035202 learned_addr learned my addr 172.21.15.40:0/397035202 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:46.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8de461640 1 --2- 172.21.15.40:0/397035202 >> v2:172.21.15.86:3300/0 conn(0x7fc8d815d180 0x7fc8d806b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:46.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8dd45f640 1 -- 172.21.15.40:0/397035202 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2065686483 0 0) 0x7fc8d80cf480 con 0x7fc8d80ad820 2023-10-16T08:38:46.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8dd45f640 1 -- 172.21.15.40:0/397035202 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc8c4003530 con 0x7fc8d80ad820 2023-10-16T08:38:46.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8dd45f640 1 -- 172.21.15.40:0/397035202 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2393035167 0 0) 0x7fc8c4003530 con 0x7fc8d80ad820 2023-10-16T08:38:46.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8dd45f640 1 -- 172.21.15.40:0/397035202 >> v2:172.21.15.86:3300/0 conn(0x7fc8d815d180 msgr2=0x7fc8d806b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:46.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8dd45f640 1 --2- 172.21.15.40:0/397035202 >> v2:172.21.15.86:3300/0 conn(0x7fc8d815d180 0x7fc8d806b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8dd45f640 1 -- 172.21.15.40:0/397035202 >> v1:172.21.15.86:6789/0 conn(0x7fc8d815d560 legacy=0x7fc8d80c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:46.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8dd45f640 1 -- 172.21.15.40:0/397035202 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d80c9290 con 0x7fc8d80ad820 2023-10-16T08:38:46.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8dd45f640 1 -- 172.21.15.40:0/397035202 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7fc8d0001a00 con 0x7fc8d80ad820 2023-10-16T08:38:46.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8dd45f640 1 -- 172.21.15.40:0/397035202 >> v1:172.21.15.40:6789/0 conn(0x7fc8d80ad820 legacy=0x7fc8d80cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:46.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8dd45f640 1 --2- 172.21.15.40:0/397035202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c40039f0 0x7fc8c4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:46.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8dd45f640 1 --2- 172.21.15.40:0/397035202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8c40047d0 0x7fc8c4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:46.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8dd45f640 1 --2- 172.21.15.40:0/397035202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c4005190 0x7fc8c400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:46.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8dd45f640 1 -- 172.21.15.40:0/397035202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d80c8fb0 con 0x7fc8c40039f0 2023-10-16T08:38:46.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8dd45f640 1 -- 172.21.15.40:0/397035202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c400c0c0 con 0x7fc8c4005190 2023-10-16T08:38:46.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8dd45f640 1 -- 172.21.15.40:0/397035202 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8c400c280 con 0x7fc8c40047d0 2023-10-16T08:38:46.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8de461640 1 --2- 172.21.15.40:0/397035202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c40039f0 0x7fc8c4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8dec62640 1 --2- 172.21.15.40:0/397035202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c4005190 0x7fc8c400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8de461640 1 -- 172.21.15.40:0/397035202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8c40047d0 msgr2=0x7fc8c4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8ddc60640 1 --2- 172.21.15.40:0/397035202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8c40047d0 0x7fc8c4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8de461640 1 --2- 172.21.15.40:0/397035202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8c40047d0 0x7fc8c4004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8de461640 1 -- 172.21.15.40:0/397035202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c4005190 msgr2=0x7fc8c400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:46.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8de461640 1 --2- 172.21.15.40:0/397035202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c4005190 0x7fc8c400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8de461640 1 -- 172.21.15.40:0/397035202 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c4003530 con 0x7fc8c40039f0 2023-10-16T08:38:46.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8dec62640 1 --2- 172.21.15.40:0/397035202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8c4005190 0x7fc8c400b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:46.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.072+0000 7fc8dec62640 1 -- 172.21.15.40:0/397035202 reap_dead start 2023-10-16T08:38:46.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.076+0000 7fc8de461640 1 --2- 172.21.15.40:0/397035202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c40039f0 0x7fc8c4003e10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009ae0 tx=0x7fc8d401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90e6ced87e92f442 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:46.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.076+0000 7fc8dd45f640 1 -- 172.21.15.40:0/397035202 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40a5640 con 0x7fc8c40039f0 2023-10-16T08:38:46.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.076+0000 7fc8dd45f640 1 -- 172.21.15.40:0/397035202 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d40a57a0 con 0x7fc8c40039f0 2023-10-16T08:38:46.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.076+0000 7fc8dd45f640 1 -- 172.21.15.40:0/397035202 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40a5b50 con 0x7fc8c40039f0 2023-10-16T08:38:46.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.076+0000 7fc8e06c5640 1 -- 172.21.15.40:0/397035202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c40039f0 msgr2=0x7fc8c4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:46.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.076+0000 7fc8e06c5640 1 --2- 172.21.15.40:0/397035202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c40039f0 0x7fc8c4003e10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fc8d4009ae0 tx=0x7fc8d401ca50 comp rx=0 tx=0).stop 2023-10-16T08:38:46.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.076+0000 7fc8e06c5640 1 -- 172.21.15.40:0/397035202 shutdown_connections 2023-10-16T08:38:46.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.076+0000 7fc8e06c5640 1 --2- 172.21.15.40:0/397035202 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c40039f0 0x7fc8c4003e10 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.076+0000 7fc8e06c5640 1 -- 172.21.15.40:0/397035202 >> 172.21.15.40:0/397035202 conn(0x7fc8d80ce960 msgr2=0x7fc8d80ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:46.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.076+0000 7fc8e06c5640 1 -- 172.21.15.40:0/397035202 shutdown_connections 2023-10-16T08:38:46.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.076+0000 7fc8e06c5640 1 -- 172.21.15.40:0/397035202 wait complete. 2023-10-16T08:38:46.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.076+0000 7fc8e06c5640 1 Processor -- start 2023-10-16T08:38:46.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.076+0000 7fc8e06c5640 1 -- start start 2023-10-16T08:38:46.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.076+0000 7fc8e06c5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c40039f0 0x7fc8d8157bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:46.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.076+0000 7fc8e06c5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8d80c81b0 0x7fc8d815b520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:46.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.076+0000 7fc8e06c5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8d80c8630 0x7fc8d815ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:46.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.076+0000 7fc8e06c5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d8158100 con 0x7fc8c40039f0 2023-10-16T08:38:46.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.076+0000 7fc8e06c5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d80c2830 con 0x7fc8d80c8630 2023-10-16T08:38:46.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8e06c5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d80c2b10 con 0x7fc8d80c81b0 2023-10-16T08:38:46.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8de461640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c40039f0 0x7fc8d8157bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:46.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8de461640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c40039f0 0x7fc8d8157bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58008/0 (socket says 172.21.15.40:58008) 2023-10-16T08:38:46.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8de461640 1 -- 172.21.15.40:0/3863212449 learned_addr learned my addr 172.21.15.40:0/3863212449 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:46.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8ddc60640 1 --2- 172.21.15.40:0/3863212449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8d80c81b0 0x7fc8d815b520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8dec62640 1 --2- 172.21.15.40:0/3863212449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8d80c8630 0x7fc8d815ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8de461640 1 -- 172.21.15.40:0/3863212449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8d80c81b0 msgr2=0x7fc8d815b520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8de461640 1 --2- 172.21.15.40:0/3863212449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8d80c81b0 0x7fc8d815b520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8de461640 1 -- 172.21.15.40:0/3863212449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8d80c8630 msgr2=0x7fc8d815ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:46.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8de461640 1 --2- 172.21.15.40:0/3863212449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8d80c8630 0x7fc8d815ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8de461640 1 -- 172.21.15.40:0/3863212449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d80c2df0 con 0x7fc8c40039f0 2023-10-16T08:38:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8ddc60640 1 --2- 172.21.15.40:0/3863212449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8d80c81b0 0x7fc8d815b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:38:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8de461640 1 --2- 172.21.15.40:0/3863212449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c40039f0 0x7fc8d8157bc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc8d401c880 tx=0x7fc8d40197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8ceffd640 1 -- 172.21.15.40:0/3863212449 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40bd070 con 0x7fc8c40039f0 2023-10-16T08:38:46.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8ceffd640 1 -- 172.21.15.40:0/3863212449 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8d4004020 con 0x7fc8c40039f0 2023-10-16T08:38:46.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8e06c5640 1 -- 172.21.15.40:0/3863212449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d80c30d0 con 0x7fc8c40039f0 2023-10-16T08:38:46.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8ceffd640 1 -- 172.21.15.40:0/3863212449 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8d40043d0 con 0x7fc8c40039f0 2023-10-16T08:38:46.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8e06c5640 1 -- 172.21.15.40:0/3863212449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d80c2490 con 0x7fc8c40039f0 2023-10-16T08:38:46.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8e06c5640 1 -- 172.21.15.40:0/3863212449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d80b2240 con 0x7fc8c40039f0 2023-10-16T08:38:46.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8ceffd640 1 -- 172.21.15.40:0/3863212449 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc8d40b4070 con 0x7fc8c40039f0 2023-10-16T08:38:46.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8ceffd640 1 --2- 172.21.15.40:0/3863212449 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fc8bc0664d0 0x7fc8bc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:46.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8ceffd640 1 -- 172.21.15.40:0/3863212449 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc8d411dc60 con 0x7fc8c40039f0 2023-10-16T08:38:46.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.080+0000 7fc8ddc60640 1 --2- 172.21.15.40:0/3863212449 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fc8bc0664d0 0x7fc8bc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:46.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.084+0000 7fc8ddc60640 1 --2- 172.21.15.40:0/3863212449 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fc8bc0664d0 0x7fc8bc068990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80017a0 tx=0x7fc8c8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:46.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.084+0000 7fc8ceffd640 1 -- 172.21.15.40:0/3863212449 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fc8d40eb170 con 0x7fc8c40039f0 2023-10-16T08:38:46.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.248+0000 7fc8e06c5640 1 -- 172.21.15.40:0/3863212449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8d804e9c0 con 0x7fc8c40039f0 2023-10-16T08:38:46.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.248+0000 7fc8ceffd640 1 -- 172.21.15.40:0/3863212449 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16458 (secure 0 0 0) 0x7fc8d40ee790 con 0x7fc8c40039f0 2023-10-16T08:38:46.254 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:46.255 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":19,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:44.984300+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:38:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.252+0000 7fc8e06c5640 1 -- 172.21.15.40:0/3863212449 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fc8bc0664d0 msgr2=0x7fc8bc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:46.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.252+0000 7fc8e06c5640 1 --2- 172.21.15.40:0/3863212449 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fc8bc0664d0 0x7fc8bc068990 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fc8c80017a0 tx=0x7fc8c8009040 comp rx=0 tx=0).stop 2023-10-16T08:38:46.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.252+0000 7fc8e06c5640 1 -- 172.21.15.40:0/3863212449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c40039f0 msgr2=0x7fc8d8157bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:46.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.252+0000 7fc8e06c5640 1 --2- 172.21.15.40:0/3863212449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c40039f0 0x7fc8d8157bc0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc8d401c880 tx=0x7fc8d40197f0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.252+0000 7fc8e06c5640 1 -- 172.21.15.40:0/3863212449 shutdown_connections 2023-10-16T08:38:46.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.252+0000 7fc8e06c5640 1 --2- 172.21.15.40:0/3863212449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8d80c8630 0x7fc8d815ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.252+0000 7fc8e06c5640 1 --2- 172.21.15.40:0/3863212449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8d80c81b0 0x7fc8d815b520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.252+0000 7fc8e06c5640 1 --2- 172.21.15.40:0/3863212449 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fc8bc0664d0 0x7fc8bc068990 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.252+0000 7fc8e06c5640 1 --2- 172.21.15.40:0/3863212449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8c40039f0 0x7fc8d8157bc0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.252+0000 7fc8e06c5640 1 -- 172.21.15.40:0/3863212449 >> 172.21.15.40:0/3863212449 conn(0x7fc8d80ce960 msgr2=0x7fc8d80cb270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:46.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.252+0000 7fc8e06c5640 1 -- 172.21.15.40:0/3863212449 shutdown_connections 2023-10-16T08:38:46.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.252+0000 7fc8e06c5640 1 -- 172.21.15.40:0/3863212449 wait complete. 2023-10-16T08:38:46.273 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:38:46.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.392+0000 7f0e05aa6640 1 Processor -- start 2023-10-16T08:38:46.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.392+0000 7f0e05aa6640 1 -- start start 2023-10-16T08:38:46.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0e05aa6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0e0015b040 0x7f0e000ca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:46.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0e05aa6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e0006c7c0 0x7f0e000cac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:46.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0e05aa6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e000adca0 con 0x7f0e000adec0 2023-10-16T08:38:46.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0e05aa6640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e000c5840 con 0x7f0e0015b040 2023-10-16T08:38:46.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0e05aa6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000cf890 con 0x7f0e0006c7c0 2023-10-16T08:38:46.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0dfeffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0e000adec0 0x7f0e000c8fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58272/0 (socket says 172.21.15.40:58272) 2023-10-16T08:38:46.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0dfeffd640 1 -- 172.21.15.40:0/2956579626 learned_addr learned my addr 172.21.15.40:0/2956579626 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:46.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0dff7fe640 1 --2- 172.21.15.40:0/2956579626 >> v2:172.21.15.40:3300/0 conn(0x7f0e0015b040 0x7f0e000ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:46.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0dfffff640 1 --2- 172.21.15.40:0/2956579626 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e0006c7c0 0x7f0e000cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:46.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0dfe7fc640 1 -- 172.21.15.40:0/2956579626 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 715502707 0 0) 0x7f0e000adca0 con 0x7f0e000adec0 2023-10-16T08:38:46.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0dfe7fc640 1 -- 172.21.15.40:0/2956579626 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0de8003530 con 0x7f0e000adec0 2023-10-16T08:38:46.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0dff7fe640 1 -- 172.21.15.40:0/2956579626 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e0006c7c0 msgr2=0x7f0e000cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:46.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0dff7fe640 1 --2- 172.21.15.40:0/2956579626 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e0006c7c0 0x7f0e000cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0dff7fe640 1 -- 172.21.15.40:0/2956579626 >> v1:172.21.15.40:6789/0 conn(0x7f0e000adec0 legacy=0x7f0e000c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:46.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0dff7fe640 1 -- 172.21.15.40:0/2956579626 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e000cf5b0 con 0x7f0e0015b040 2023-10-16T08:38:46.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0dfffff640 1 --2- 172.21.15.40:0/2956579626 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e0006c7c0 0x7f0e000cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:38:46.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0dff7fe640 1 --2- 172.21.15.40:0/2956579626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e0015b040 0x7f0e000ca740 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0df00097c0 tx=0x7f0df001c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefcd3d915816110 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:46.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0dfe7fc640 1 -- 172.21.15.40:0/2956579626 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df00ab600 con 0x7f0e0015b040 2023-10-16T08:38:46.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0dfe7fc640 1 -- 172.21.15.40:0/2956579626 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df00a7050 con 0x7f0e0015b040 2023-10-16T08:38:46.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0dfe7fc640 1 -- 172.21.15.40:0/2956579626 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df00ab980 con 0x7f0e0015b040 2023-10-16T08:38:46.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0e05aa6640 1 -- 172.21.15.40:0/2956579626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e0015b040 msgr2=0x7f0e000ca740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:46.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0e05aa6640 1 --2- 172.21.15.40:0/2956579626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e0015b040 0x7f0e000ca740 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0df00097c0 tx=0x7f0df001c330 comp rx=0 tx=0).stop 2023-10-16T08:38:46.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0e05aa6640 1 -- 172.21.15.40:0/2956579626 shutdown_connections 2023-10-16T08:38:46.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0e05aa6640 1 --2- 172.21.15.40:0/2956579626 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e0006c7c0 0x7f0e000cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0e05aa6640 1 --2- 172.21.15.40:0/2956579626 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e0015b040 0x7f0e000ca740 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0e05aa6640 1 -- 172.21.15.40:0/2956579626 >> 172.21.15.40:0/2956579626 conn(0x7f0e000ce960 msgr2=0x7f0e000ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:46.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0e05aa6640 1 -- 172.21.15.40:0/2956579626 shutdown_connections 2023-10-16T08:38:46.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.396+0000 7f0e05aa6640 1 -- 172.21.15.40:0/2956579626 wait complete. 2023-10-16T08:38:46.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0e05aa6640 1 Processor -- start 2023-10-16T08:38:46.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0e05aa6640 1 -- start start 2023-10-16T08:38:46.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0e05aa6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e0006c7c0 0x7f0e001f6040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:46.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0e05aa6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e000adec0 0x7f0e000c2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:46.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0e05aa6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e0015b040 0x7f0e000c4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:46.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0e05aa6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e001f5bb0 con 0x7f0e0015b040 2023-10-16T08:38:46.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0e05aa6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000c30f0 con 0x7f0e000adec0 2023-10-16T08:38:46.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0e05aa6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e000c4f60 con 0x7f0e0006c7c0 2023-10-16T08:38:46.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0dfffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e0015b040 0x7f0e000c4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:46.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0dfffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e0015b040 0x7f0e000c4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58022/0 (socket says 172.21.15.40:58022) 2023-10-16T08:38:46.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0dfffff640 1 -- 172.21.15.40:0/2882551770 learned_addr learned my addr 172.21.15.40:0/2882551770 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:46.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0dff7fe640 1 --2- 172.21.15.40:0/2882551770 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e0006c7c0 0x7f0e001f6040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:46.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0dfeffd640 1 --2- 172.21.15.40:0/2882551770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e000adec0 0x7f0e000c2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:46.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0dfffff640 1 -- 172.21.15.40:0/2882551770 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e0006c7c0 msgr2=0x7f0e001f6040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:46.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0dfffff640 1 --2- 172.21.15.40:0/2882551770 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e0006c7c0 0x7f0e001f6040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0dfffff640 1 -- 172.21.15.40:0/2882551770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e000adec0 msgr2=0x7f0e000c2bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:46.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0dfffff640 1 --2- 172.21.15.40:0/2882551770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e000adec0 0x7f0e000c2bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0dfffff640 1 -- 172.21.15.40:0/2882551770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df0009420 con 0x7f0e0015b040 2023-10-16T08:38:46.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0dfffff640 1 --2- 172.21.15.40:0/2882551770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e0015b040 0x7f0e000c4530 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0df8007d20 tx=0x7f0df800d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:46.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0de7fff640 1 -- 172.21.15.40:0/2882551770 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df8015400 con 0x7f0e0015b040 2023-10-16T08:38:46.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0de7fff640 1 -- 172.21.15.40:0/2882551770 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df800f040 con 0x7f0e0015b040 2023-10-16T08:38:46.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0de7fff640 1 -- 172.21.15.40:0/2882551770 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df80157a0 con 0x7f0e0015b040 2023-10-16T08:38:46.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0e05aa6640 1 -- 172.21.15.40:0/2882551770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e000c5240 con 0x7f0e0015b040 2023-10-16T08:38:46.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0e05aa6640 1 -- 172.21.15.40:0/2882551770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e001f5840 con 0x7f0e0015b040 2023-10-16T08:38:46.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.400+0000 7f0e05aa6640 1 -- 172.21.15.40:0/2882551770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e001f5a00 con 0x7f0e0015b040 2023-10-16T08:38:46.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.404+0000 7f0de7fff640 1 -- 172.21.15.40:0/2882551770 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0df801e3b0 con 0x7f0e0015b040 2023-10-16T08:38:46.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.404+0000 7f0de7fff640 1 --2- 172.21.15.40:0/2882551770 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0dec0664a0 0x7f0dec068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:46.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.404+0000 7f0de7fff640 1 -- 172.21.15.40:0/2882551770 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0df8084150 con 0x7f0e0015b040 2023-10-16T08:38:46.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.404+0000 7f0dff7fe640 1 --2- 172.21.15.40:0/2882551770 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0dec0664a0 0x7f0dec068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:46.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.404+0000 7f0dff7fe640 1 --2- 172.21.15.40:0/2882551770 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0dec0664a0 0x7f0dec068960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0df00ad000 tx=0x7f0df00a6000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:46.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.408+0000 7f0de7fff640 1 -- 172.21.15.40:0/2882551770 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f0df8051530 con 0x7f0e0015b040 2023-10-16T08:38:46.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.568+0000 7f0e05aa6640 1 -- 172.21.15.40:0/2882551770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e0004e9c0 con 0x7f0e0015b040 2023-10-16T08:38:46.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.568+0000 7f0de7fff640 1 -- 172.21.15.40:0/2882551770 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+16458 (secure 0 0 0) 0x7f0df8054b50 con 0x7f0e0015b040 2023-10-16T08:38:46.574 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:46.574 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":19,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:44.984300+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:38:46.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.572+0000 7f0e05aa6640 1 -- 172.21.15.40:0/2882551770 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0dec0664a0 msgr2=0x7f0dec068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:46.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.572+0000 7f0e05aa6640 1 --2- 172.21.15.40:0/2882551770 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0dec0664a0 0x7f0dec068960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0df00ad000 tx=0x7f0df00a6000 comp rx=0 tx=0).stop 2023-10-16T08:38:46.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.572+0000 7f0e05aa6640 1 -- 172.21.15.40:0/2882551770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e0015b040 msgr2=0x7f0e000c4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:46.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.572+0000 7f0e05aa6640 1 --2- 172.21.15.40:0/2882551770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e0015b040 0x7f0e000c4530 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f0df8007d20 tx=0x7f0df800d4a0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.572+0000 7f0e05aa6640 1 -- 172.21.15.40:0/2882551770 shutdown_connections 2023-10-16T08:38:46.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.572+0000 7f0e05aa6640 1 --2- 172.21.15.40:0/2882551770 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0dec0664a0 0x7f0dec068960 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.572+0000 7f0e05aa6640 1 --2- 172.21.15.40:0/2882551770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0e0015b040 0x7f0e000c4530 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.572+0000 7f0e05aa6640 1 --2- 172.21.15.40:0/2882551770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e000adec0 0x7f0e000c2bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.572+0000 7f0e05aa6640 1 --2- 172.21.15.40:0/2882551770 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0e0006c7c0 0x7f0e001f6040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.572+0000 7f0e05aa6640 1 -- 172.21.15.40:0/2882551770 >> 172.21.15.40:0/2882551770 conn(0x7f0e000ce960 msgr2=0x7f0e0015a160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:46.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.572+0000 7f0e05aa6640 1 -- 172.21.15.40:0/2882551770 shutdown_connections 2023-10-16T08:38:46.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.572+0000 7f0e05aa6640 1 -- 172.21.15.40:0/2882551770 wait complete. 2023-10-16T08:38:46.594 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-16T08:38:46.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.716+0000 7f493d86b640 1 Processor -- start 2023-10-16T08:38:46.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.716+0000 7f493d86b640 1 -- start start 2023-10-16T08:38:46.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.716+0000 7f493d86b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49380adad0 0x7f49380b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:46.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.716+0000 7f493d86b640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f49380cae10 0x7f493815a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:46.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.716+0000 7f493d86b640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49380c98b0 con 0x7f49380adeb0 2023-10-16T08:38:46.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.716+0000 7f493d86b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49380b47c0 con 0x7f49380adad0 2023-10-16T08:38:46.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.716+0000 7f493d86b640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49380c8f80 con 0x7f49380cae10 2023-10-16T08:38:46.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.716+0000 7f4936ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49380adad0 0x7f49380b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:46.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.716+0000 7f4936ffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f49380adad0 0x7f49380b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58030/0 (socket says 172.21.15.40:58030) 2023-10-16T08:38:46.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.716+0000 7f4936ffd640 1 -- 172.21.15.40:0/3570193758 learned_addr learned my addr 172.21.15.40:0/3570193758 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:46.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.716+0000 7f49377fe640 1 --2- 172.21.15.40:0/3570193758 >> v2:172.21.15.86:3300/0 conn(0x7f49380cae10 0x7f493815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:46.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.716+0000 7f4936ffd640 1 -- 172.21.15.40:0/3570193758 >> v2:172.21.15.86:3300/0 conn(0x7f49380cae10 msgr2=0x7f493815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:46.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.716+0000 7f4936ffd640 1 --2- 172.21.15.40:0/3570193758 >> v2:172.21.15.86:3300/0 conn(0x7f49380cae10 0x7f493815a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.716+0000 7f4936ffd640 1 -- 172.21.15.40:0/3570193758 >> v1:172.21.15.86:6789/0 conn(0x7f49380adeb0 legacy=0x7f49380ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:46.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.716+0000 7f4936ffd640 1 -- 172.21.15.40:0/3570193758 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49380c9260 con 0x7f49380adad0 2023-10-16T08:38:46.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.720+0000 7f4936ffd640 1 --2- 172.21.15.40:0/3570193758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49380adad0 0x7f49380b41e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f493806d3d0 tx=0x7f493001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486237712ec42a82 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:46.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.720+0000 7f4935ffb640 1 -- 172.21.15.40:0/3570193758 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49300a5880 con 0x7f49380adad0 2023-10-16T08:38:46.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.720+0000 7f4935ffb640 1 -- 172.21.15.40:0/3570193758 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49300a59e0 con 0x7f49380adad0 2023-10-16T08:38:46.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.720+0000 7f4935ffb640 1 -- 172.21.15.40:0/3570193758 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49300a5d70 con 0x7f49380adad0 2023-10-16T08:38:46.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.720+0000 7f493d86b640 1 -- 172.21.15.40:0/3570193758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49380adad0 msgr2=0x7f49380b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:46.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.720+0000 7f493d86b640 1 --2- 172.21.15.40:0/3570193758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49380adad0 0x7f49380b41e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f493806d3d0 tx=0x7f493001c9a0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.720+0000 7f493d86b640 1 -- 172.21.15.40:0/3570193758 shutdown_connections 2023-10-16T08:38:46.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.720+0000 7f493d86b640 1 --2- 172.21.15.40:0/3570193758 >> v2:172.21.15.86:3300/0 conn(0x7f49380cae10 0x7f493815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.720+0000 7f493d86b640 1 --2- 172.21.15.40:0/3570193758 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49380adad0 0x7f49380b41e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.720+0000 7f493d86b640 1 -- 172.21.15.40:0/3570193758 >> 172.21.15.40:0/3570193758 conn(0x7f49380ce890 msgr2=0x7f49380ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:46.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.720+0000 7f493d86b640 1 -- 172.21.15.40:0/3570193758 shutdown_connections 2023-10-16T08:38:46.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.720+0000 7f493d86b640 1 -- 172.21.15.40:0/3570193758 wait complete. 2023-10-16T08:38:46.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.720+0000 7f493d86b640 1 Processor -- start 2023-10-16T08:38:46.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.720+0000 7f493d86b640 1 -- start start 2023-10-16T08:38:46.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f493d86b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49380adad0 0x7f493814c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:46.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f493d86b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49380adeb0 0x7f4938150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:46.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f493d86b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49380cae10 0x7f49380ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:46.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f493d86b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f493814c980 con 0x7f49380adeb0 2023-10-16T08:38:46.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f493d86b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4938151750 con 0x7f49380cae10 2023-10-16T08:38:46.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f493d86b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4938151a30 con 0x7f49380adad0 2023-10-16T08:38:46.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f49367fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49380adeb0 0x7f4938150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:46.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f49367fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49380adeb0 0x7f4938150d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58040/0 (socket says 172.21.15.40:58040) 2023-10-16T08:38:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f49367fc640 1 -- 172.21.15.40:0/3318872675 learned_addr learned my addr 172.21.15.40:0/3318872675 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f49377fe640 1 --2- 172.21.15.40:0/3318872675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49380cae10 0x7f49380ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f4936ffd640 1 --2- 172.21.15.40:0/3318872675 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49380adad0 0x7f493814c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f49367fc640 1 -- 172.21.15.40:0/3318872675 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49380adad0 msgr2=0x7f493814c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f49367fc640 1 --2- 172.21.15.40:0/3318872675 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49380adad0 0x7f493814c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f49367fc640 1 -- 172.21.15.40:0/3318872675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49380cae10 msgr2=0x7f49380ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:46.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f49367fc640 1 --2- 172.21.15.40:0/3318872675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49380cae10 0x7f49380ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f49367fc640 1 -- 172.21.15.40:0/3318872675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4930009b30 con 0x7f49380adeb0 2023-10-16T08:38:46.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f49377fe640 1 --2- 172.21.15.40:0/3318872675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49380cae10 0x7f49380ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:46.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f49367fc640 1 --2- 172.21.15.40:0/3318872675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49380adeb0 0x7f4938150d00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f492000c4d0 tx=0x7f492000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:46.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f492ffff640 1 -- 172.21.15.40:0/3318872675 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49200052d0 con 0x7f49380adeb0 2023-10-16T08:38:46.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f492ffff640 1 -- 172.21.15.40:0/3318872675 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4920005430 con 0x7f49380adeb0 2023-10-16T08:38:46.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f493d86b640 1 -- 172.21.15.40:0/3318872675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f493815b250 con 0x7f49380adeb0 2023-10-16T08:38:46.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f492ffff640 1 -- 172.21.15.40:0/3318872675 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4920012830 con 0x7f49380adeb0 2023-10-16T08:38:46.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f493d86b640 1 -- 172.21.15.40:0/3318872675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4938151ff0 con 0x7f49380adeb0 2023-10-16T08:38:46.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f493d86b640 1 -- 172.21.15.40:0/3318872675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4938067a20 con 0x7f49380adeb0 2023-10-16T08:38:46.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f492ffff640 1 -- 172.21.15.40:0/3318872675 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4920012990 con 0x7f49380adeb0 2023-10-16T08:38:46.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f492ffff640 1 --2- 172.21.15.40:0/3318872675 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f49180664a0 0x7f4918068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:46.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f492ffff640 1 -- 172.21.15.40:0/3318872675 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4920083990 con 0x7f49380adeb0 2023-10-16T08:38:46.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.724+0000 7f4936ffd640 1 --2- 172.21.15.40:0/3318872675 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f49180664a0 0x7f4918068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:46.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.728+0000 7f4936ffd640 1 --2- 172.21.15.40:0/3318872675 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f49180664a0 0x7f4918068960 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f49300ae000 tx=0x7f49300a7070 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:46.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.728+0000 7f492ffff640 1 -- 172.21.15.40:0/3318872675 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f4920050e10 con 0x7f49380adeb0 2023-10-16T08:38:46.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.920+0000 7f493d86b640 1 -- 172.21.15.40:0/3318872675 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f493804e9c0 con 0x7f49380adeb0 2023-10-16T08:38:46.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.944+0000 7f492ffff640 1 -- 172.21.15.40:0/3318872675 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v5) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4920054430 con 0x7f49380adeb0 2023-10-16T08:38:46.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.948+0000 7f493d86b640 1 -- 172.21.15.40:0/3318872675 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f49180664a0 msgr2=0x7f4918068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:46.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.948+0000 7f493d86b640 1 --2- 172.21.15.40:0/3318872675 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f49180664a0 0x7f4918068960 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f49300ae000 tx=0x7f49300a7070 comp rx=0 tx=0).stop 2023-10-16T08:38:46.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.948+0000 7f493d86b640 1 -- 172.21.15.40:0/3318872675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49380adeb0 msgr2=0x7f4938150d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:46.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.948+0000 7f493d86b640 1 --2- 172.21.15.40:0/3318872675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49380adeb0 0x7f4938150d00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f492000c4d0 tx=0x7f492000c990 comp rx=0 tx=0).stop 2023-10-16T08:38:46.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.948+0000 7f493d86b640 1 -- 172.21.15.40:0/3318872675 shutdown_connections 2023-10-16T08:38:46.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.948+0000 7f493d86b640 1 --2- 172.21.15.40:0/3318872675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f49380cae10 0x7f49380ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.948+0000 7f493d86b640 1 --2- 172.21.15.40:0/3318872675 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f49180664a0 0x7f4918068960 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.948+0000 7f493d86b640 1 --2- 172.21.15.40:0/3318872675 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f49380adeb0 0x7f4938150d00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.948+0000 7f493d86b640 1 --2- 172.21.15.40:0/3318872675 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f49380adad0 0x7f493814c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:46.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.948+0000 7f493d86b640 1 -- 172.21.15.40:0/3318872675 >> 172.21.15.40:0/3318872675 conn(0x7f49380ce890 msgr2=0x7f493815e6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:46.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.948+0000 7f493d86b640 1 -- 172.21.15.40:0/3318872675 shutdown_connections 2023-10-16T08:38:46.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:46.948+0000 7f493d86b640 1 -- 172.21.15.40:0/3318872675 wait complete. 2023-10-16T08:38:46.967 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-10-16T08:38:46.968 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-16T08:38:47.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.092+0000 7f2801659640 1 Processor -- start 2023-10-16T08:38:47.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.092+0000 7f2801659640 1 -- start start 2023-10-16T08:38:47.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.092+0000 7f2801659640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f27fc0adad0 0x7f27fc0b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:47.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.092+0000 7f2801659640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27fc0c98b0 con 0x7f27fc0adeb0 2023-10-16T08:38:47.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.092+0000 7f2801659640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27fc0b47c0 con 0x7f27fc0cae10 2023-10-16T08:38:47.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.092+0000 7f2801659640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27fc0c8f80 con 0x7f27fc0adad0 2023-10-16T08:38:47.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.092+0000 7f27fa7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f27fc0adeb0 0x7f27fc0ca700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58276/0 (socket says 172.21.15.40:58276) 2023-10-16T08:38:47.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.092+0000 7f27faffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f27fc0adad0 0x7f27fc0b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:47.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.092+0000 7f27fa7fc640 1 -- 172.21.15.40:0/4163759415 learned_addr learned my addr 172.21.15.40:0/4163759415 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:47.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.092+0000 7f27faffd640 1 -- 172.21.15.40:0/4163759415 >> v1:172.21.15.86:6789/0 conn(0x7f27fc0cae10 legacy=0x7f27fc15a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:47.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.096+0000 7f27faffd640 1 -- 172.21.15.40:0/4163759415 >> v1:172.21.15.40:6789/0 conn(0x7f27fc0adeb0 legacy=0x7f27fc0ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:47.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.096+0000 7f27faffd640 1 -- 172.21.15.40:0/4163759415 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc0c9260 con 0x7f27fc0adad0 2023-10-16T08:38:47.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.096+0000 7f27f9ffb640 1 -- 172.21.15.40:0/4163759415 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 960678031 0 0) 0x7f27fc0c98b0 con 0x7f27fc0adeb0 2023-10-16T08:38:47.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.096+0000 7f27faffd640 1 --2- 172.21.15.40:0/4163759415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27fc0adad0 0x7f27fc0b41e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f27e40097c0 tx=0x7f27e401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b04269146efa9f6a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:47.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.096+0000 7f27f9ffb640 1 -- 172.21.15.40:0/4163759415 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e40ab670 con 0x7f27fc0adad0 2023-10-16T08:38:47.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.096+0000 7f27f9ffb640 1 -- 172.21.15.40:0/4163759415 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e40ab7d0 con 0x7f27fc0adad0 2023-10-16T08:38:47.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.096+0000 7f27f9ffb640 1 -- 172.21.15.40:0/4163759415 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e40abb60 con 0x7f27fc0adad0 2023-10-16T08:38:47.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.096+0000 7f2801659640 1 -- 172.21.15.40:0/4163759415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27fc0adad0 msgr2=0x7f27fc0b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:47.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.096+0000 7f2801659640 1 --2- 172.21.15.40:0/4163759415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27fc0adad0 0x7f27fc0b41e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f27e40097c0 tx=0x7f27e401c450 comp rx=0 tx=0).stop 2023-10-16T08:38:47.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.096+0000 7f2801659640 1 -- 172.21.15.40:0/4163759415 shutdown_connections 2023-10-16T08:38:47.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.096+0000 7f2801659640 1 --2- 172.21.15.40:0/4163759415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27fc0adad0 0x7f27fc0b41e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:47.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.096+0000 7f2801659640 1 -- 172.21.15.40:0/4163759415 >> 172.21.15.40:0/4163759415 conn(0x7f27fc0ce890 msgr2=0x7f27fc0ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:47.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.096+0000 7f2801659640 1 -- 172.21.15.40:0/4163759415 shutdown_connections 2023-10-16T08:38:47.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.096+0000 7f2801659640 1 -- 172.21.15.40:0/4163759415 wait complete. 2023-10-16T08:38:47.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f2801659640 1 Processor -- start 2023-10-16T08:38:47.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f2801659640 1 -- start start 2023-10-16T08:38:47.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f2801659640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27fc0adad0 0x7f27fc147c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:47.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f2801659640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27fc0adeb0 0x7f27fc0c44f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:47.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f2801659640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27fc0cae10 0x7f27fc0c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:47.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f2801659640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc14a7b0 con 0x7f27fc0adad0 2023-10-16T08:38:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f2801659640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc14cb00 con 0x7f27fc0adeb0 2023-10-16T08:38:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f2801659640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27fc14c560 con 0x7f27fc0cae10 2023-10-16T08:38:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f27faffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27fc0adad0 0x7f27fc147c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:47.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f27faffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27fc0adad0 0x7f27fc147c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58058/0 (socket says 172.21.15.40:58058) 2023-10-16T08:38:47.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f27faffd640 1 -- 172.21.15.40:0/2676654114 learned_addr learned my addr 172.21.15.40:0/2676654114 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:47.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f27fa7fc640 1 --2- 172.21.15.40:0/2676654114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27fc0adeb0 0x7f27fc0c44f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:47.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f27faffd640 1 -- 172.21.15.40:0/2676654114 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27fc0cae10 msgr2=0x7f27fc0c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:47.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f27faffd640 1 --2- 172.21.15.40:0/2676654114 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27fc0cae10 0x7f27fc0c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:47.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f27faffd640 1 -- 172.21.15.40:0/2676654114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27fc0adeb0 msgr2=0x7f27fc0c44f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:47.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f27faffd640 1 --2- 172.21.15.40:0/2676654114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27fc0adeb0 0x7f27fc0c44f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:47.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f27faffd640 1 -- 172.21.15.40:0/2676654114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e4009420 con 0x7f27fc0adad0 2023-10-16T08:38:47.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f27faffd640 1 --2- 172.21.15.40:0/2676654114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27fc0adad0 0x7f27fc147c70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f27e40ad040 tx=0x7f27e401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:47.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f27f37fe640 1 -- 172.21.15.40:0/2676654114 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e40b5810 con 0x7f27fc0adad0 2023-10-16T08:38:47.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f27f37fe640 1 -- 172.21.15.40:0/2676654114 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e40b5990 con 0x7f27fc0adad0 2023-10-16T08:38:47.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f2801659640 1 -- 172.21.15.40:0/2676654114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27fc1533a0 con 0x7f27fc0adad0 2023-10-16T08:38:47.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f27f37fe640 1 -- 172.21.15.40:0/2676654114 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e40b5d40 con 0x7f27fc0adad0 2023-10-16T08:38:47.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f2801659640 1 -- 172.21.15.40:0/2676654114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27fc153920 con 0x7f27fc0adad0 2023-10-16T08:38:47.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f2801659640 1 -- 172.21.15.40:0/2676654114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27fc067a20 con 0x7f27fc0adad0 2023-10-16T08:38:47.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.100+0000 7f27f37fe640 1 -- 172.21.15.40:0/2676654114 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f27e40c83f0 con 0x7f27fc0adad0 2023-10-16T08:38:47.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.104+0000 7f27f37fe640 1 --2- 172.21.15.40:0/2676654114 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f27dc0664a0 0x7f27dc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:47.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.104+0000 7f27f37fe640 1 -- 172.21.15.40:0/2676654114 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27e411ee10 con 0x7f27fc0adad0 2023-10-16T08:38:47.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.104+0000 7f27fa7fc640 1 --2- 172.21.15.40:0/2676654114 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f27dc0664a0 0x7f27dc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:47.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.104+0000 7f27fa7fc640 1 --2- 172.21.15.40:0/2676654114 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f27dc0664a0 0x7f27dc068960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f27fc1585b0 tx=0x7f27ec009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:47.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.108+0000 7f27f37fe640 1 -- 172.21.15.40:0/2676654114 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f27e40ec440 con 0x7f27fc0adad0 2023-10-16T08:38:47.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.300+0000 7f2801659640 1 -- 172.21.15.40:0/2676654114 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f27fc04e9c0 con 0x7f27fc0adad0 2023-10-16T08:38:47.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.948+0000 7f27f37fe640 1 -- 172.21.15.40:0/2676654114 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v6) v1 ==== 122+0+0 (secure 0 0 0) 0x7f27e40efa60 con 0x7f27fc0adad0 2023-10-16T08:38:47.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.952+0000 7f2801659640 1 -- 172.21.15.40:0/2676654114 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f27dc0664a0 msgr2=0x7f27dc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:47.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.952+0000 7f2801659640 1 --2- 172.21.15.40:0/2676654114 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f27dc0664a0 0x7f27dc068960 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f27fc1585b0 tx=0x7f27ec009040 comp rx=0 tx=0).stop 2023-10-16T08:38:47.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.952+0000 7f2801659640 1 -- 172.21.15.40:0/2676654114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27fc0adad0 msgr2=0x7f27fc147c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:47.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.952+0000 7f2801659640 1 --2- 172.21.15.40:0/2676654114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27fc0adad0 0x7f27fc147c70 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f27e40ad040 tx=0x7f27e401ac90 comp rx=0 tx=0).stop 2023-10-16T08:38:47.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.952+0000 7f2801659640 1 -- 172.21.15.40:0/2676654114 shutdown_connections 2023-10-16T08:38:47.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.952+0000 7f2801659640 1 --2- 172.21.15.40:0/2676654114 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27fc0cae10 0x7f27fc0c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:47.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.952+0000 7f2801659640 1 --2- 172.21.15.40:0/2676654114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27fc0adeb0 0x7f27fc0c44f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:47.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.952+0000 7f2801659640 1 --2- 172.21.15.40:0/2676654114 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f27dc0664a0 0x7f27dc068960 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:47.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.952+0000 7f2801659640 1 --2- 172.21.15.40:0/2676654114 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27fc0adad0 0x7f27fc147c70 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:47.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.952+0000 7f2801659640 1 -- 172.21.15.40:0/2676654114 >> 172.21.15.40:0/2676654114 conn(0x7f27fc0ce890 msgr2=0x7f27fc0c59f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:47.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.952+0000 7f2801659640 1 -- 172.21.15.40:0/2676654114 shutdown_connections 2023-10-16T08:38:47.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:47.952+0000 7f2801659640 1 -- 172.21.15.40:0/2676654114 wait complete. 2023-10-16T08:38:47.978 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-10-16T08:38:47.978 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:38:48.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.100+0000 7f6518946640 1 Processor -- start 2023-10-16T08:38:48.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.100+0000 7f6518946640 1 -- start start 2023-10-16T08:38:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.104+0000 7f6518946640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f651415b040 0x7f65140ca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.104+0000 7f6518946640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f651406c7c0 0x7f65140cac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:48.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.104+0000 7f6518946640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65140adca0 con 0x7f65140adec0 2023-10-16T08:38:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.104+0000 7f6518946640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65140c5840 con 0x7f651406c7c0 2023-10-16T08:38:48.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.104+0000 7f6518946640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65140cf890 con 0x7f651415b040 2023-10-16T08:38:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.104+0000 7f6512d9d640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f651406c7c0 0x7f65140cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.104+0000 7f651259c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f651415b040 0x7f65140ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.104+0000 7f6511d9b640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f65140adec0 0x7f65140c8fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.40:47146/0 (socket says 172.21.15.40:47146) 2023-10-16T08:38:48.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.104+0000 7f6511d9b640 1 -- 172.21.15.40:0/2365418217 learned_addr learned my addr 172.21.15.40:0/2365418217 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:48.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.104+0000 7f651159a640 1 -- 172.21.15.40:0/2365418217 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3520468014 0 0) 0x7f65140adca0 con 0x7f65140adec0 2023-10-16T08:38:48.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.104+0000 7f651159a640 1 -- 172.21.15.40:0/2365418217 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64fc003530 con 0x7f65140adec0 2023-10-16T08:38:48.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.104+0000 7f651259c640 1 -- 172.21.15.40:0/2365418217 >> v2:172.21.15.86:3300/0 conn(0x7f651406c7c0 msgr2=0x7f65140cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:48.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.104+0000 7f651259c640 1 --2- 172.21.15.40:0/2365418217 >> v2:172.21.15.86:3300/0 conn(0x7f651406c7c0 0x7f65140cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.104+0000 7f651259c640 1 -- 172.21.15.40:0/2365418217 >> v1:172.21.15.86:6789/0 conn(0x7f65140adec0 legacy=0x7f65140c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.104+0000 7f651259c640 1 -- 172.21.15.40:0/2365418217 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65140cf5b0 con 0x7f651415b040 2023-10-16T08:38:48.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.104+0000 7f6512d9d640 1 --2- 172.21.15.40:0/2365418217 >> v2:172.21.15.86:3300/0 conn(0x7f651406c7c0 0x7f65140cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:38:48.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f651259c640 1 --2- 172.21.15.40:0/2365418217 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f651415b040 0x7f65140ca740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f650c0097c0 tx=0x7f650c01c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1faeff74b54f18f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:48.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f651159a640 1 -- 172.21.15.40:0/2365418217 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c0ab600 con 0x7f651415b040 2023-10-16T08:38:48.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f651159a640 1 -- 172.21.15.40:0/2365418217 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650c0a7050 con 0x7f651415b040 2023-10-16T08:38:48.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f651159a640 1 -- 172.21.15.40:0/2365418217 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650c0ab980 con 0x7f651415b040 2023-10-16T08:38:48.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6518946640 1 -- 172.21.15.40:0/2365418217 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f651415b040 msgr2=0x7f65140ca740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:48.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6518946640 1 --2- 172.21.15.40:0/2365418217 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f651415b040 0x7f65140ca740 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f650c0097c0 tx=0x7f650c01c330 comp rx=0 tx=0).stop 2023-10-16T08:38:48.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6518946640 1 -- 172.21.15.40:0/2365418217 shutdown_connections 2023-10-16T08:38:48.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6518946640 1 --2- 172.21.15.40:0/2365418217 >> v2:172.21.15.86:3300/0 conn(0x7f651406c7c0 0x7f65140cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6518946640 1 --2- 172.21.15.40:0/2365418217 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f651415b040 0x7f65140ca740 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6518946640 1 -- 172.21.15.40:0/2365418217 >> 172.21.15.40:0/2365418217 conn(0x7f65140ce960 msgr2=0x7f65140ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:48.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6518946640 1 -- 172.21.15.40:0/2365418217 shutdown_connections 2023-10-16T08:38:48.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6518946640 1 -- 172.21.15.40:0/2365418217 wait complete. 2023-10-16T08:38:48.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6518946640 1 Processor -- start 2023-10-16T08:38:48.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6518946640 1 -- start start 2023-10-16T08:38:48.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6518946640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f651406c7c0 0x7f65140c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:48.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6518946640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65140adec0 0x7f65140c4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:48.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6518946640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f651415b040 0x7f651415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6518946640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f651415c580 con 0x7f651415b040 2023-10-16T08:38:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6518946640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65140c25c0 con 0x7f651406c7c0 2023-10-16T08:38:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6518946640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65140c28a0 con 0x7f65140adec0 2023-10-16T08:38:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6512d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f651415b040 0x7f651415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:48.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6512d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f651415b040 0x7f651415bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58072/0 (socket says 172.21.15.40:58072) 2023-10-16T08:38:48.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6512d9d640 1 -- 172.21.15.40:0/1488310904 learned_addr learned my addr 172.21.15.40:0/1488310904 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:48.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6511d9b640 1 --2- 172.21.15.40:0/1488310904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65140adec0 0x7f65140c4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:48.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f651259c640 1 --2- 172.21.15.40:0/1488310904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f651406c7c0 0x7f65140c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:48.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6512d9d640 1 -- 172.21.15.40:0/1488310904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65140adec0 msgr2=0x7f65140c4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:48.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6512d9d640 1 --2- 172.21.15.40:0/1488310904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65140adec0 0x7f65140c4cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6512d9d640 1 -- 172.21.15.40:0/1488310904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f651406c7c0 msgr2=0x7f65140c30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:48.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6512d9d640 1 --2- 172.21.15.40:0/1488310904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f651406c7c0 0x7f65140c30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6512d9d640 1 -- 172.21.15.40:0/1488310904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6504009420 con 0x7f651415b040 2023-10-16T08:38:48.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6512d9d640 1 --2- 172.21.15.40:0/1488310904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f651415b040 0x7f651415bb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f650400e7c0 tx=0x7f650400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:48.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.108+0000 7f6502ffd640 1 -- 172.21.15.40:0/1488310904 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65040095b0 con 0x7f651415b040 2023-10-16T08:38:48.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.112+0000 7f6502ffd640 1 -- 172.21.15.40:0/1488310904 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f650400c940 con 0x7f651415b040 2023-10-16T08:38:48.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.112+0000 7f6518946640 1 -- 172.21.15.40:0/1488310904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f650c009420 con 0x7f651415b040 2023-10-16T08:38:48.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.112+0000 7f6502ffd640 1 -- 172.21.15.40:0/1488310904 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f650400ccf0 con 0x7f651415b040 2023-10-16T08:38:48.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.112+0000 7f6518946640 1 -- 172.21.15.40:0/1488310904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f651415c860 con 0x7f651415b040 2023-10-16T08:38:48.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.112+0000 7f6518946640 1 -- 172.21.15.40:0/1488310904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65140c2dc0 con 0x7f651415b040 2023-10-16T08:38:48.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.112+0000 7f6502ffd640 1 -- 172.21.15.40:0/1488310904 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6504013470 con 0x7f651415b040 2023-10-16T08:38:48.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.112+0000 7f6502ffd640 1 --2- 172.21.15.40:0/1488310904 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f64f40664a0 0x7f64f4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:48.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.112+0000 7f6502ffd640 1 -- 172.21.15.40:0/1488310904 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6504083f30 con 0x7f651415b040 2023-10-16T08:38:48.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.112+0000 7f651259c640 1 --2- 172.21.15.40:0/1488310904 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f64f40664a0 0x7f64f4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:48.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.112+0000 7f651259c640 1 --2- 172.21.15.40:0/1488310904 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f64f40664a0 0x7f64f4068960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f650c0ad000 tx=0x7f650c0a6000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:48.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.116+0000 7f6502ffd640 1 -- 172.21.15.40:0/1488310904 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f65140c2dc0 con 0x7f651415b040 2023-10-16T08:38:48.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.308+0000 7f6518946640 1 -- 172.21.15.40:0/1488310904 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65140c2dc0 con 0x7f651415b040 2023-10-16T08:38:48.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.312+0000 7f6502ffd640 1 -- 172.21.15.40:0/1488310904 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4898 (secure 0 0 0) 0x7f65140c2dc0 con 0x7f651415b040 2023-10-16T08:38:48.316 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:48.316 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/32529848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":32529848},{"type":"v1","addr":"172.21.15.86:6837","nonce":32529848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2044121591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2044121591},{"type":"v1","addr":"172.21.15.86:6836","nonce":2044121591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1275484997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1275484997},{"type":"v1","addr":"172.21.15.40:6833","nonce":1275484997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:38:44.984772+0000","modified":"2023-10-16T08:38:47.305255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/148507680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":148507680},{"type":"v1","addr":"172.21.15.40:6835","nonce":148507680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T08:38:48.316 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 6 2023-10-16T08:38:48.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.312+0000 7f6518946640 1 -- 172.21.15.40:0/1488310904 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f64f40664a0 msgr2=0x7f64f4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:48.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.312+0000 7f6518946640 1 --2- 172.21.15.40:0/1488310904 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f64f40664a0 0x7f64f4068960 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f650c0ad000 tx=0x7f650c0a6000 comp rx=0 tx=0).stop 2023-10-16T08:38:48.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.312+0000 7f6518946640 1 -- 172.21.15.40:0/1488310904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f651415b040 msgr2=0x7f651415bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:48.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.312+0000 7f6518946640 1 --2- 172.21.15.40:0/1488310904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f651415b040 0x7f651415bb30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f650400e7c0 tx=0x7f650400ec80 comp rx=0 tx=0).stop 2023-10-16T08:38:48.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.312+0000 7f6518946640 1 -- 172.21.15.40:0/1488310904 shutdown_connections 2023-10-16T08:38:48.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.312+0000 7f6518946640 1 --2- 172.21.15.40:0/1488310904 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f64f40664a0 0x7f64f4068960 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.312+0000 7f6518946640 1 --2- 172.21.15.40:0/1488310904 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f651415b040 0x7f651415bb30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.312+0000 7f6518946640 1 --2- 172.21.15.40:0/1488310904 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65140adec0 0x7f65140c4cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.312+0000 7f6518946640 1 --2- 172.21.15.40:0/1488310904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f651406c7c0 0x7f65140c30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.312+0000 7f6518946640 1 -- 172.21.15.40:0/1488310904 >> 172.21.15.40:0/1488310904 conn(0x7f65140ce960 msgr2=0x7f65141571c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:48.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.312+0000 7f6518946640 1 -- 172.21.15.40:0/1488310904 shutdown_connections 2023-10-16T08:38:48.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.312+0000 7f6518946640 1 -- 172.21.15.40:0/1488310904 wait complete. 2023-10-16T08:38:48.335 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:38:48.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.448+0000 7f2af35d9640 1 Processor -- start 2023-10-16T08:38:48.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.448+0000 7f2af35d9640 1 -- start start 2023-10-16T08:38:48.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.448+0000 7f2af35d9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2aec15d560 0x7f2aec06b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:48.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.448+0000 7f2af35d9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aec0cf480 con 0x7f2aec0ad820 2023-10-16T08:38:48.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.448+0000 7f2af35d9640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2aec0cf760 con 0x7f2aec15d180 2023-10-16T08:38:48.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.448+0000 7f2af35d9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2aec0c8fb0 con 0x7f2aec15d560 2023-10-16T08:38:48.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.448+0000 7f2af0b74640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2aec15d560 0x7f2aec06b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:48.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.448+0000 7f2af1b76640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2aec0ad820 0x7f2aec0cd220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58292/0 (socket says 172.21.15.40:58292) 2023-10-16T08:38:48.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.448+0000 7f2af1b76640 1 -- 172.21.15.40:0/389046218 learned_addr learned my addr 172.21.15.40:0/389046218 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:48.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.448+0000 7f2af0b74640 1 -- 172.21.15.40:0/389046218 >> v1:172.21.15.86:6789/0 conn(0x7f2aec15d180 legacy=0x7f2aec0c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:48.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.448+0000 7f2af0b74640 1 -- 172.21.15.40:0/389046218 >> v1:172.21.15.40:6789/0 conn(0x7f2aec0ad820 legacy=0x7f2aec0cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:48.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.448+0000 7f2af0b74640 1 -- 172.21.15.40:0/389046218 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec0c9290 con 0x7f2aec15d560 2023-10-16T08:38:48.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.448+0000 7f2ae3fff640 1 -- 172.21.15.40:0/389046218 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3509189983 0 0) 0x7f2aec0cf480 con 0x7f2aec0ad820 2023-10-16T08:38:48.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.452+0000 7f2af0b74640 1 --2- 172.21.15.40:0/389046218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2aec15d560 0x7f2aec06b7e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0097c0 tx=0x7f2adc01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f20895205ace874c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:48.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.452+0000 7f2ae3fff640 1 -- 172.21.15.40:0/389046218 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0ab660 con 0x7f2aec15d560 2023-10-16T08:38:48.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.452+0000 7f2ae3fff640 1 -- 172.21.15.40:0/389046218 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2adc0ab7c0 con 0x7f2aec15d560 2023-10-16T08:38:48.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.452+0000 7f2ae3fff640 1 -- 172.21.15.40:0/389046218 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2adc0abb50 con 0x7f2aec15d560 2023-10-16T08:38:48.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.452+0000 7f2af35d9640 1 -- 172.21.15.40:0/389046218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2aec15d560 msgr2=0x7f2aec06b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:48.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.452+0000 7f2af35d9640 1 --2- 172.21.15.40:0/389046218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2aec15d560 0x7f2aec06b7e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0097c0 tx=0x7f2adc01c480 comp rx=0 tx=0).stop 2023-10-16T08:38:48.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.452+0000 7f2af35d9640 1 -- 172.21.15.40:0/389046218 shutdown_connections 2023-10-16T08:38:48.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.452+0000 7f2af35d9640 1 --2- 172.21.15.40:0/389046218 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2aec15d560 0x7f2aec06b7e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.452+0000 7f2af35d9640 1 -- 172.21.15.40:0/389046218 >> 172.21.15.40:0/389046218 conn(0x7f2aec0ce960 msgr2=0x7f2aec0ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:48.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.452+0000 7f2af35d9640 1 -- 172.21.15.40:0/389046218 shutdown_connections 2023-10-16T08:38:48.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.452+0000 7f2af35d9640 1 -- 172.21.15.40:0/389046218 wait complete. 2023-10-16T08:38:48.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.452+0000 7f2af35d9640 1 Processor -- start 2023-10-16T08:38:48.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.452+0000 7f2af35d9640 1 -- start start 2023-10-16T08:38:48.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af35d9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2aec0ad820 0x7f2aec14fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:48.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af35d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aec15d180 0x7f2aec156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:48.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af35d9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2aec15d560 0x7f2aec156d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:48.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af35d9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec153b40 con 0x7f2aec0ad820 2023-10-16T08:38:48.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af35d9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec153e20 con 0x7f2aec15d180 2023-10-16T08:38:48.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af35d9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aec15c700 con 0x7f2aec15d560 2023-10-16T08:38:48.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af1375640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2aec0ad820 0x7f2aec14fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:48.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af1375640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2aec0ad820 0x7f2aec14fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58096/0 (socket says 172.21.15.40:58096) 2023-10-16T08:38:48.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af1375640 1 -- 172.21.15.40:0/788163449 learned_addr learned my addr 172.21.15.40:0/788163449 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:48.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af1b76640 1 --2- 172.21.15.40:0/788163449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2aec15d560 0x7f2aec156d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:48.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af0b74640 1 --2- 172.21.15.40:0/788163449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aec15d180 0x7f2aec156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:48.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af1375640 1 -- 172.21.15.40:0/788163449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2aec15d560 msgr2=0x7f2aec156d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:48.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af1375640 1 --2- 172.21.15.40:0/788163449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2aec15d560 0x7f2aec156d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af1375640 1 -- 172.21.15.40:0/788163449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aec15d180 msgr2=0x7f2aec156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:48.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af1375640 1 --2- 172.21.15.40:0/788163449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aec15d180 0x7f2aec156820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af1375640 1 -- 172.21.15.40:0/788163449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2adc009420 con 0x7f2aec0ad820 2023-10-16T08:38:48.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af1375640 1 --2- 172.21.15.40:0/788163449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2aec0ad820 0x7f2aec14fcd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800c530 tx=0x7f2ae800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:48.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2ae1ffb640 1 -- 172.21.15.40:0/788163449 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8005220 con 0x7f2aec0ad820 2023-10-16T08:38:48.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2ae1ffb640 1 -- 172.21.15.40:0/788163449 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ae8005380 con 0x7f2aec0ad820 2023-10-16T08:38:48.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2ae1ffb640 1 -- 172.21.15.40:0/788163449 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae8007270 con 0x7f2aec0ad820 2023-10-16T08:38:48.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af35d9640 1 -- 172.21.15.40:0/788163449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2aec1573c0 con 0x7f2aec0ad820 2023-10-16T08:38:48.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af35d9640 1 -- 172.21.15.40:0/788163449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2aec157980 con 0x7f2aec0ad820 2023-10-16T08:38:48.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af35d9640 1 -- 172.21.15.40:0/788163449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2aec157b70 con 0x7f2aec0ad820 2023-10-16T08:38:48.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2ae1ffb640 1 -- 172.21.15.40:0/788163449 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2ae80073d0 con 0x7f2aec0ad820 2023-10-16T08:38:48.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2ae1ffb640 1 --2- 172.21.15.40:0/788163449 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f2ad00664a0 0x7f2ad0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:48.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2ae1ffb640 1 -- 172.21.15.40:0/788163449 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2ae80836f0 con 0x7f2aec0ad820 2023-10-16T08:38:48.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.456+0000 7f2af0b74640 1 --2- 172.21.15.40:0/788163449 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f2ad00664a0 0x7f2ad0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:48.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.460+0000 7f2af0b74640 1 --2- 172.21.15.40:0/788163449 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f2ad00664a0 0x7f2ad0068960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0af000 tx=0x7f2adc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:48.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.460+0000 7f2ae1ffb640 1 -- 172.21.15.40:0/788163449 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f2ae8050c00 con 0x7f2aec0ad820 2023-10-16T08:38:48.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.620+0000 7f2af35d9640 1 -- 172.21.15.40:0/788163449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2aec04e9c0 con 0x7f2aec0ad820 2023-10-16T08:38:48.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.620+0000 7f2ae1ffb640 1 -- 172.21.15.40:0/788163449 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f2ae8054220 con 0x7f2aec0ad820 2023-10-16T08:38:48.627 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:48.627 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:46.611098+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:38:48.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.624+0000 7f2af35d9640 1 -- 172.21.15.40:0/788163449 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f2ad00664a0 msgr2=0x7f2ad0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:48.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.624+0000 7f2af35d9640 1 --2- 172.21.15.40:0/788163449 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f2ad00664a0 0x7f2ad0068960 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2adc0af000 tx=0x7f2adc008040 comp rx=0 tx=0).stop 2023-10-16T08:38:48.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.624+0000 7f2af35d9640 1 -- 172.21.15.40:0/788163449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2aec0ad820 msgr2=0x7f2aec14fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:48.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.624+0000 7f2af35d9640 1 --2- 172.21.15.40:0/788163449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2aec0ad820 0x7f2aec14fcd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800c530 tx=0x7f2ae800c9f0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.624+0000 7f2af35d9640 1 -- 172.21.15.40:0/788163449 shutdown_connections 2023-10-16T08:38:48.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.624+0000 7f2af35d9640 1 --2- 172.21.15.40:0/788163449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2aec15d560 0x7f2aec156d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.624+0000 7f2af35d9640 1 --2- 172.21.15.40:0/788163449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aec15d180 0x7f2aec156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.624+0000 7f2af35d9640 1 --2- 172.21.15.40:0/788163449 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f2ad00664a0 0x7f2ad0068960 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.624+0000 7f2af35d9640 1 --2- 172.21.15.40:0/788163449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2aec0ad820 0x7f2aec14fcd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.624+0000 7f2af35d9640 1 -- 172.21.15.40:0/788163449 >> 172.21.15.40:0/788163449 conn(0x7f2aec0ce960 msgr2=0x7f2aec0ca4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:48.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.624+0000 7f2af35d9640 1 -- 172.21.15.40:0/788163449 shutdown_connections 2023-10-16T08:38:48.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.624+0000 7f2af35d9640 1 -- 172.21.15.40:0/788163449 wait complete. 2023-10-16T08:38:48.645 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-16T08:38:48.646 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:38:48.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.764+0000 7fd767725640 1 Processor -- start 2023-10-16T08:38:48.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.764+0000 7fd767725640 1 -- start start 2023-10-16T08:38:48.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd767725640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd76015cce0 0x7fd7600b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:48.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd767725640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7600b7f80 con 0x7fd7600ce070 2023-10-16T08:38:48.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd767725640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7600b7370 con 0x7fd7600b6e10 2023-10-16T08:38:48.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd767725640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7600b7650 con 0x7fd76015cce0 2023-10-16T08:38:48.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd765cc2640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd7600ce070 0x7fd7600d0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58308/0 (socket says 172.21.15.40:58308) 2023-10-16T08:38:48.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd765cc2640 1 -- 172.21.15.40:0/1568561889 learned_addr learned my addr 172.21.15.40:0/1568561889 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:48.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd7654c1640 1 --2- 172.21.15.40:0/1568561889 >> v2:172.21.15.86:3300/0 conn(0x7fd76015cce0 0x7fd7600b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:48.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd753fff640 1 -- 172.21.15.40:0/1568561889 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 806382640 0 0) 0x7fd7600b7f80 con 0x7fd7600ce070 2023-10-16T08:38:48.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd753fff640 1 -- 172.21.15.40:0/1568561889 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd74c003530 con 0x7fd7600ce070 2023-10-16T08:38:48.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd753fff640 1 -- 172.21.15.40:0/1568561889 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3220856507 0 0) 0x7fd74c003530 con 0x7fd7600ce070 2023-10-16T08:38:48.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd753fff640 1 -- 172.21.15.40:0/1568561889 >> v2:172.21.15.86:3300/0 conn(0x7fd76015cce0 msgr2=0x7fd7600b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:48.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd753fff640 1 --2- 172.21.15.40:0/1568561889 >> v2:172.21.15.86:3300/0 conn(0x7fd76015cce0 0x7fd7600b68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd753fff640 1 -- 172.21.15.40:0/1568561889 >> v1:172.21.15.86:6789/0 conn(0x7fd7600b6e10 legacy=0x7fd7600cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:48.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd753fff640 1 -- 172.21.15.40:0/1568561889 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7600b7930 con 0x7fd7600ce070 2023-10-16T08:38:48.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd753fff640 1 -- 172.21.15.40:0/1568561889 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7fd758001680 con 0x7fd7600ce070 2023-10-16T08:38:48.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd753fff640 1 -- 172.21.15.40:0/1568561889 >> v1:172.21.15.40:6789/0 conn(0x7fd7600ce070 legacy=0x7fd7600d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:48.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd753fff640 1 --2- 172.21.15.40:0/1568561889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd74c0039f0 0x7fd74c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:48.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd753fff640 1 --2- 172.21.15.40:0/1568561889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd74c0047d0 0x7fd74c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:48.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd753fff640 1 --2- 172.21.15.40:0/1568561889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd74c005190 0x7fd74c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:48.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd753fff640 1 -- 172.21.15.40:0/1568561889 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7600b7650 con 0x7fd74c005190 2023-10-16T08:38:48.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd753fff640 1 -- 172.21.15.40:0/1568561889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c00c0c0 con 0x7fd74c0047d0 2023-10-16T08:38:48.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd753fff640 1 -- 172.21.15.40:0/1568561889 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd74c00c280 con 0x7fd74c0039f0 2023-10-16T08:38:48.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd765cc2640 1 --2- 172.21.15.40:0/1568561889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd74c005190 0x7fd74c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:48.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd764cc0640 1 --2- 172.21.15.40:0/1568561889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd74c0047d0 0x7fd74c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:48.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd7654c1640 1 --2- 172.21.15.40:0/1568561889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd74c0039f0 0x7fd74c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:48.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd765cc2640 1 -- 172.21.15.40:0/1568561889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd74c0039f0 msgr2=0x7fd74c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:48.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd765cc2640 1 --2- 172.21.15.40:0/1568561889 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd74c0039f0 0x7fd74c003e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd765cc2640 1 -- 172.21.15.40:0/1568561889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd74c0047d0 msgr2=0x7fd74c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:48.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd765cc2640 1 --2- 172.21.15.40:0/1568561889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd74c0047d0 0x7fd74c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.768+0000 7fd765cc2640 1 -- 172.21.15.40:0/1568561889 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd74c003530 con 0x7fd74c005190 2023-10-16T08:38:48.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd765cc2640 1 -- 172.21.15.40:0/1568561889 reap_dead start 2023-10-16T08:38:48.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd765cc2640 1 --2- 172.21.15.40:0/1568561889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd74c005190 0x7fd74c00b990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd75800c210 tx=0x7fd75801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe17bcaa986be134 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:48.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd753fff640 1 -- 172.21.15.40:0/1568561889 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7580a5b20 con 0x7fd74c005190 2023-10-16T08:38:48.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd753fff640 1 -- 172.21.15.40:0/1568561889 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7580a5c80 con 0x7fd74c005190 2023-10-16T08:38:48.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd753fff640 1 -- 172.21.15.40:0/1568561889 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7580058e0 con 0x7fd74c005190 2023-10-16T08:38:48.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd767725640 1 -- 172.21.15.40:0/1568561889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd74c005190 msgr2=0x7fd74c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:48.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd767725640 1 --2- 172.21.15.40:0/1568561889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd74c005190 0x7fd74c00b990 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd75800c210 tx=0x7fd75801cc20 comp rx=0 tx=0).stop 2023-10-16T08:38:48.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd767725640 1 -- 172.21.15.40:0/1568561889 shutdown_connections 2023-10-16T08:38:48.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd767725640 1 --2- 172.21.15.40:0/1568561889 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd74c005190 0x7fd74c00b990 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd767725640 1 -- 172.21.15.40:0/1568561889 >> 172.21.15.40:0/1568561889 conn(0x7fd76006a290 msgr2=0x7fd76006a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:48.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd767725640 1 -- 172.21.15.40:0/1568561889 shutdown_connections 2023-10-16T08:38:48.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd767725640 1 -- 172.21.15.40:0/1568561889 wait complete. 2023-10-16T08:38:48.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd767725640 1 Processor -- start 2023-10-16T08:38:48.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd767725640 1 -- start start 2023-10-16T08:38:48.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd767725640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd74c005190 0x7fd7600c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:48.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd767725640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7600d2c70 0x7fd7600d30f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:48.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd767725640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd760157f60 0x7fd7601583e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:48.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd767725640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7600d3630 con 0x7fd7600d2c70 2023-10-16T08:38:48.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd767725640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7600ca4e0 con 0x7fd74c005190 2023-10-16T08:38:48.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd767725640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7600c6d80 con 0x7fd760157f60 2023-10-16T08:38:48.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd764cc0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7600d2c70 0x7fd7600d30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:48.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd764cc0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7600d2c70 0x7fd7600d30f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58112/0 (socket says 172.21.15.40:58112) 2023-10-16T08:38:48.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd764cc0640 1 -- 172.21.15.40:0/20367394 learned_addr learned my addr 172.21.15.40:0/20367394 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:48.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd7654c1640 1 --2- 172.21.15.40:0/20367394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd74c005190 0x7fd7600c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:48.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd765cc2640 1 --2- 172.21.15.40:0/20367394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd760157f60 0x7fd7601583e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:48.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd764cc0640 1 -- 172.21.15.40:0/20367394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd760157f60 msgr2=0x7fd7601583e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:48.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd764cc0640 1 --2- 172.21.15.40:0/20367394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd760157f60 0x7fd7601583e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd764cc0640 1 -- 172.21.15.40:0/20367394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd74c005190 msgr2=0x7fd7600c67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:48.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd764cc0640 1 --2- 172.21.15.40:0/20367394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd74c005190 0x7fd7600c67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.772+0000 7fd764cc0640 1 -- 172.21.15.40:0/20367394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7600c9e90 con 0x7fd7600d2c70 2023-10-16T08:38:48.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.776+0000 7fd764cc0640 1 --2- 172.21.15.40:0/20367394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7600d2c70 0x7fd7600d30f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00e7b0 tx=0x7fd75c00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:48.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.776+0000 7fd751ffb640 1 -- 172.21.15.40:0/20367394 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c003b90 con 0x7fd7600d2c70 2023-10-16T08:38:48.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.776+0000 7fd751ffb640 1 -- 172.21.15.40:0/20367394 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd75c003cf0 con 0x7fd7600d2c70 2023-10-16T08:38:48.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.776+0000 7fd767725640 1 -- 172.21.15.40:0/20367394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7600ca170 con 0x7fd7600d2c70 2023-10-16T08:38:48.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.776+0000 7fd751ffb640 1 -- 172.21.15.40:0/20367394 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd75c0052f0 con 0x7fd7600d2c70 2023-10-16T08:38:48.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.776+0000 7fd767725640 1 -- 172.21.15.40:0/20367394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd760158920 con 0x7fd7600d2c70 2023-10-16T08:38:48.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.776+0000 7fd767725640 1 -- 172.21.15.40:0/20367394 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd760158b10 con 0x7fd7600d2c70 2023-10-16T08:38:48.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.776+0000 7fd751ffb640 1 -- 172.21.15.40:0/20367394 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd75c005450 con 0x7fd7600d2c70 2023-10-16T08:38:48.795 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.776+0000 7fd751ffb640 1 --2- 172.21.15.40:0/20367394 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd7440664d0 0x7fd744068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:48.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.776+0000 7fd751ffb640 1 -- 172.21.15.40:0/20367394 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd75c083ea0 con 0x7fd7600d2c70 2023-10-16T08:38:48.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.776+0000 7fd7654c1640 1 --2- 172.21.15.40:0/20367394 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd7440664d0 0x7fd744068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:48.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.776+0000 7fd7654c1640 1 --2- 172.21.15.40:0/20367394 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd7440664d0 0x7fd744068990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd7600d0fc0 tx=0x7fd754008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:48.796 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.780+0000 7fd751ffb640 1 -- 172.21.15.40:0/20367394 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fd75c0513b0 con 0x7fd7600d2c70 2023-10-16T08:38:48.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.936+0000 7fd767725640 1 -- 172.21.15.40:0/20367394 --> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd7600b8b70 con 0x7fd7440664d0 2023-10-16T08:38:48.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.960+0000 7fd751ffb640 1 -- 172.21.15.40:0/20367394 <== mgr.4111 v2:172.21.15.86:6832/1188970899 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264546 (secure 0 0 0) 0x7fd7540128f0 con 0x7fd7440664d0 2023-10-16T08:38:48.964 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:48.966 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-16T08:38:48.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.960+0000 7fd767725640 1 -- 172.21.15.40:0/20367394 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd7440664d0 msgr2=0x7fd744068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:48.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.960+0000 7fd767725640 1 --2- 172.21.15.40:0/20367394 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd7440664d0 0x7fd744068990 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd7600d0fc0 tx=0x7fd754008040 comp rx=0 tx=0).stop 2023-10-16T08:38:48.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.960+0000 7fd767725640 1 -- 172.21.15.40:0/20367394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7600d2c70 msgr2=0x7fd7600d30f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:48.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.960+0000 7fd767725640 1 --2- 172.21.15.40:0/20367394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7600d2c70 0x7fd7600d30f0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd75c00e7b0 tx=0x7fd75c00ec70 comp rx=0 tx=0).stop 2023-10-16T08:38:48.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.960+0000 7fd767725640 1 -- 172.21.15.40:0/20367394 shutdown_connections 2023-10-16T08:38:48.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.960+0000 7fd767725640 1 --2- 172.21.15.40:0/20367394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd760157f60 0x7fd7601583e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.960+0000 7fd767725640 1 --2- 172.21.15.40:0/20367394 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd7440664d0 0x7fd744068990 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.960+0000 7fd767725640 1 --2- 172.21.15.40:0/20367394 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd7600d2c70 0x7fd7600d30f0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.960+0000 7fd767725640 1 --2- 172.21.15.40:0/20367394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd74c005190 0x7fd7600c67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:48.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.960+0000 7fd767725640 1 -- 172.21.15.40:0/20367394 >> 172.21.15.40:0/20367394 conn(0x7fd76006a290 msgr2=0x7fd76014f7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:48.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.960+0000 7fd767725640 1 -- 172.21.15.40:0/20367394 shutdown_connections 2023-10-16T08:38:48.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:48.960+0000 7fd767725640 1 -- 172.21.15.40:0/20367394 wait complete. 2023-10-16T08:38:48.983 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-16T08:38:48.938600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219016,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755704,"statfs":{"total":773094113280,"available":772869840896,"internally_reserved":0,"allocated":2891776,"data_stored":1752740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":63,"apply_latency_ms":63,"commit_latency_ns":63000000,"apply_latency_ns":63000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.341363"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931534+0000","last_change":"2023-10-16T08:38:45.650619+0000","last_active":"2023-10-16T08:38:46.931534+0000","last_peered":"2023-10-16T08:38:46.931534+0000","last_clean":"2023-10-16T08:38:46.931534+0000","last_became_active":"2023-10-16T08:38:45.649704+0000","last_became_peered":"2023-10-16T08:38:45.649704+0000","last_unstale":"2023-10-16T08:38:46.931534+0000","last_undegraded":"2023-10-16T08:38:46.931534+0000","last_fullsized":"2023-10-16T08:38:46.931534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:58:51.599709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626698+0000","last_change":"2023-10-16T08:38:45.657659+0000","last_active":"2023-10-16T08:38:46.626698+0000","last_peered":"2023-10-16T08:38:46.626698+0000","last_clean":"2023-10-16T08:38:46.626698+0000","last_became_active":"2023-10-16T08:38:45.656884+0000","last_became_peered":"2023-10-16T08:38:45.656884+0000","last_unstale":"2023-10-16T08:38:46.626698+0000","last_undegraded":"2023-10-16T08:38:46.626698+0000","last_fullsized":"2023-10-16T08:38:46.626698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926261+0000","last_change":"2023-10-16T08:38:45.630125+0000","last_active":"2023-10-16T08:38:46.926261+0000","last_peered":"2023-10-16T08:38:46.926261+0000","last_clean":"2023-10-16T08:38:46.926261+0000","last_became_active":"2023-10-16T08:38:45.629297+0000","last_became_peered":"2023-10-16T08:38:45.629297+0000","last_unstale":"2023-10-16T08:38:46.926261+0000","last_undegraded":"2023-10-16T08:38:46.926261+0000","last_fullsized":"2023-10-16T08:38:46.926261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931028+0000","last_change":"2023-10-16T08:38:45.675996+0000","last_active":"2023-10-16T08:38:46.931028+0000","last_peered":"2023-10-16T08:38:46.931028+0000","last_clean":"2023-10-16T08:38:46.931028+0000","last_became_active":"2023-10-16T08:38:45.675327+0000","last_became_peered":"2023-10-16T08:38:45.675327+0000","last_unstale":"2023-10-16T08:38:46.931028+0000","last_undegraded":"2023-10-16T08:38:46.931028+0000","last_fullsized":"2023-10-16T08:38:46.931028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:33.190218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930419+0000","last_change":"2023-10-16T08:38:45.653550+0000","last_active":"2023-10-16T08:38:46.930419+0000","last_peered":"2023-10-16T08:38:46.930419+0000","last_clean":"2023-10-16T08:38:46.930419+0000","last_became_active":"2023-10-16T08:38:45.652808+0000","last_became_peered":"2023-10-16T08:38:45.652808+0000","last_unstale":"2023-10-16T08:38:46.930419+0000","last_undegraded":"2023-10-16T08:38:46.930419+0000","last_fullsized":"2023-10-16T08:38:46.930419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930519+0000","last_change":"2023-10-16T08:38:45.675086+0000","last_active":"2023-10-16T08:38:46.930519+0000","last_peered":"2023-10-16T08:38:46.930519+0000","last_clean":"2023-10-16T08:38:46.930519+0000","last_became_active":"2023-10-16T08:38:45.671146+0000","last_became_peered":"2023-10-16T08:38:45.671146+0000","last_unstale":"2023-10-16T08:38:46.930519+0000","last_undegraded":"2023-10-16T08:38:46.930519+0000","last_fullsized":"2023-10-16T08:38:46.930519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931145+0000","last_change":"2023-10-16T08:38:45.652595+0000","last_active":"2023-10-16T08:38:46.931145+0000","last_peered":"2023-10-16T08:38:46.931145+0000","last_clean":"2023-10-16T08:38:46.931145+0000","last_became_active":"2023-10-16T08:38:45.651848+0000","last_became_peered":"2023-10-16T08:38:45.651848+0000","last_unstale":"2023-10-16T08:38:46.931145+0000","last_undegraded":"2023-10-16T08:38:46.931145+0000","last_fullsized":"2023-10-16T08:38:46.931145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:13.453321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628116+0000","last_change":"2023-10-16T08:38:45.668858+0000","last_active":"2023-10-16T08:38:46.628116+0000","last_peered":"2023-10-16T08:38:46.628116+0000","last_clean":"2023-10-16T08:38:46.628116+0000","last_became_active":"2023-10-16T08:38:45.664702+0000","last_became_peered":"2023-10-16T08:38:45.664702+0000","last_unstale":"2023-10-16T08:38:46.628116+0000","last_undegraded":"2023-10-16T08:38:46.628116+0000","last_fullsized":"2023-10-16T08:38:46.628116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931734+0000","last_change":"2023-10-16T08:38:45.667037+0000","last_active":"2023-10-16T08:38:46.931734+0000","last_peered":"2023-10-16T08:38:46.931734+0000","last_clean":"2023-10-16T08:38:46.931734+0000","last_became_active":"2023-10-16T08:38:45.666474+0000","last_became_peered":"2023-10-16T08:38:45.666474+0000","last_unstale":"2023-10-16T08:38:46.931734+0000","last_undegraded":"2023-10-16T08:38:46.931734+0000","last_fullsized":"2023-10-16T08:38:46.931734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:11:02.634006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633786+0000","last_change":"2023-10-16T08:38:45.669859+0000","last_active":"2023-10-16T08:38:46.633786+0000","last_peered":"2023-10-16T08:38:46.633786+0000","last_clean":"2023-10-16T08:38:46.633786+0000","last_became_active":"2023-10-16T08:38:45.668278+0000","last_became_peered":"2023-10-16T08:38:45.668278+0000","last_unstale":"2023-10-16T08:38:46.633786+0000","last_undegraded":"2023-10-16T08:38:46.633786+0000","last_fullsized":"2023-10-16T08:38:46.633786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633464+0000","last_change":"2023-10-16T08:38:45.664567+0000","last_active":"2023-10-16T08:38:46.633464+0000","last_peered":"2023-10-16T08:38:46.633464+0000","last_clean":"2023-10-16T08:38:46.633464+0000","last_became_active":"2023-10-16T08:38:45.663813+0000","last_became_peered":"2023-10-16T08:38:45.663813+0000","last_unstale":"2023-10-16T08:38:46.633464+0000","last_undegraded":"2023-10-16T08:38:46.633464+0000","last_fullsized":"2023-10-16T08:38:46.633464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.932310+0000","last_change":"2023-10-16T08:38:45.675925+0000","last_active":"2023-10-16T08:38:46.932310+0000","last_peered":"2023-10-16T08:38:46.932310+0000","last_clean":"2023-10-16T08:38:46.932310+0000","last_became_active":"2023-10-16T08:38:45.675320+0000","last_became_peered":"2023-10-16T08:38:45.675320+0000","last_unstale":"2023-10-16T08:38:46.932310+0000","last_undegraded":"2023-10-16T08:38:46.932310+0000","last_fullsized":"2023-10-16T08:38:46.932310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:11.991837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628294+0000","last_change":"2023-10-16T08:38:45.668056+0000","last_active":"2023-10-16T08:38:46.628294+0000","last_peered":"2023-10-16T08:38:46.628294+0000","last_clean":"2023-10-16T08:38:46.628294+0000","last_became_active":"2023-10-16T08:38:45.667428+0000","last_became_peered":"2023-10-16T08:38:45.667428+0000","last_unstale":"2023-10-16T08:38:46.628294+0000","last_undegraded":"2023-10-16T08:38:46.628294+0000","last_fullsized":"2023-10-16T08:38:46.628294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.925984+0000","last_change":"2023-10-16T08:38:45.637518+0000","last_active":"2023-10-16T08:38:46.925984+0000","last_peered":"2023-10-16T08:38:46.925984+0000","last_clean":"2023-10-16T08:38:46.925984+0000","last_became_active":"2023-10-16T08:38:45.636785+0000","last_became_peered":"2023-10-16T08:38:45.636785+0000","last_unstale":"2023-10-16T08:38:46.925984+0000","last_undegraded":"2023-10-16T08:38:46.925984+0000","last_fullsized":"2023-10-16T08:38:46.925984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:11.870171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626028+0000","last_change":"2023-10-16T08:38:45.656433+0000","last_active":"2023-10-16T08:38:46.626028+0000","last_peered":"2023-10-16T08:38:46.626028+0000","last_clean":"2023-10-16T08:38:46.626028+0000","last_became_active":"2023-10-16T08:38:45.655395+0000","last_became_peered":"2023-10-16T08:38:45.655395+0000","last_unstale":"2023-10-16T08:38:46.626028+0000","last_undegraded":"2023-10-16T08:38:46.626028+0000","last_fullsized":"2023-10-16T08:38:46.626028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.929381+0000","last_change":"2023-10-16T08:38:45.664192+0000","last_active":"2023-10-16T08:38:46.929381+0000","last_peered":"2023-10-16T08:38:46.929381+0000","last_clean":"2023-10-16T08:38:46.929381+0000","last_became_active":"2023-10-16T08:38:45.663312+0000","last_became_peered":"2023-10-16T08:38:45.663312+0000","last_unstale":"2023-10-16T08:38:46.929381+0000","last_undegraded":"2023-10-16T08:38:46.929381+0000","last_fullsized":"2023-10-16T08:38:46.929381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930671+0000","last_change":"2023-10-16T08:38:45.679894+0000","last_active":"2023-10-16T08:38:46.930671+0000","last_peered":"2023-10-16T08:38:46.930671+0000","last_clean":"2023-10-16T08:38:46.930671+0000","last_became_active":"2023-10-16T08:38:45.676912+0000","last_became_peered":"2023-10-16T08:38:45.676912+0000","last_unstale":"2023-10-16T08:38:46.930671+0000","last_undegraded":"2023-10-16T08:38:46.930671+0000","last_fullsized":"2023-10-16T08:38:46.930671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927699+0000","last_change":"2023-10-16T08:38:45.631488+0000","last_active":"2023-10-16T08:38:46.927699+0000","last_peered":"2023-10-16T08:38:46.927699+0000","last_clean":"2023-10-16T08:38:46.927699+0000","last_became_active":"2023-10-16T08:38:45.630641+0000","last_became_peered":"2023-10-16T08:38:45.630641+0000","last_unstale":"2023-10-16T08:38:46.927699+0000","last_undegraded":"2023-10-16T08:38:46.927699+0000","last_fullsized":"2023-10-16T08:38:46.927699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:11.991837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931188+0000","last_change":"2023-10-16T08:38:45.650087+0000","last_active":"2023-10-16T08:38:46.931188+0000","last_peered":"2023-10-16T08:38:46.931188+0000","last_clean":"2023-10-16T08:38:46.931188+0000","last_became_active":"2023-10-16T08:38:45.649475+0000","last_became_peered":"2023-10-16T08:38:45.649475+0000","last_unstale":"2023-10-16T08:38:46.931188+0000","last_undegraded":"2023-10-16T08:38:46.931188+0000","last_fullsized":"2023-10-16T08:38:46.931188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:26.292137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633283+0000","last_change":"2023-10-16T08:38:45.667336+0000","last_active":"2023-10-16T08:38:46.633283+0000","last_peered":"2023-10-16T08:38:46.633283+0000","last_clean":"2023-10-16T08:38:46.633283+0000","last_became_active":"2023-10-16T08:38:45.665291+0000","last_became_peered":"2023-10-16T08:38:45.665291+0000","last_unstale":"2023-10-16T08:38:46.633283+0000","last_undegraded":"2023-10-16T08:38:46.633283+0000","last_fullsized":"2023-10-16T08:38:46.633283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926490+0000","last_change":"2023-10-16T08:38:45.654036+0000","last_active":"2023-10-16T08:38:46.926490+0000","last_peered":"2023-10-16T08:38:46.926490+0000","last_clean":"2023-10-16T08:38:46.926490+0000","last_became_active":"2023-10-16T08:38:45.652412+0000","last_became_peered":"2023-10-16T08:38:45.652412+0000","last_unstale":"2023-10-16T08:38:46.926490+0000","last_undegraded":"2023-10-16T08:38:46.926490+0000","last_fullsized":"2023-10-16T08:38:46.926490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930858+0000","last_change":"2023-10-16T08:38:45.677644+0000","last_active":"2023-10-16T08:38:46.930858+0000","last_peered":"2023-10-16T08:38:46.930858+0000","last_clean":"2023-10-16T08:38:46.930858+0000","last_became_active":"2023-10-16T08:38:45.676844+0000","last_became_peered":"2023-10-16T08:38:45.676844+0000","last_unstale":"2023-10-16T08:38:46.930858+0000","last_undegraded":"2023-10-16T08:38:46.930858+0000","last_fullsized":"2023-10-16T08:38:46.930858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628178+0000","last_change":"2023-10-16T08:38:45.663771+0000","last_active":"2023-10-16T08:38:46.628178+0000","last_peered":"2023-10-16T08:38:46.628178+0000","last_clean":"2023-10-16T08:38:46.628178+0000","last_became_active":"2023-10-16T08:38:45.662944+0000","last_became_peered":"2023-10-16T08:38:45.662944+0000","last_unstale":"2023-10-16T08:38:46.628178+0000","last_undegraded":"2023-10-16T08:38:46.628178+0000","last_fullsized":"2023-10-16T08:38:46.628178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.929737+0000","last_change":"2023-10-16T08:38:45.667398+0000","last_active":"2023-10-16T08:38:46.929737+0000","last_peered":"2023-10-16T08:38:46.929737+0000","last_clean":"2023-10-16T08:38:46.929737+0000","last_became_active":"2023-10-16T08:38:45.666641+0000","last_became_peered":"2023-10-16T08:38:45.666641+0000","last_unstale":"2023-10-16T08:38:46.929737+0000","last_undegraded":"2023-10-16T08:38:46.929737+0000","last_fullsized":"2023-10-16T08:38:46.929737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628798+0000","last_change":"2023-10-16T08:38:45.670526+0000","last_active":"2023-10-16T08:38:46.628798+0000","last_peered":"2023-10-16T08:38:46.628798+0000","last_clean":"2023-10-16T08:38:46.628798+0000","last_became_active":"2023-10-16T08:38:45.669734+0000","last_became_peered":"2023-10-16T08:38:45.669734+0000","last_unstale":"2023-10-16T08:38:46.628798+0000","last_undegraded":"2023-10-16T08:38:46.628798+0000","last_fullsized":"2023-10-16T08:38:46.628798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633809+0000","last_change":"2023-10-16T08:38:45.670429+0000","last_active":"2023-10-16T08:38:46.633809+0000","last_peered":"2023-10-16T08:38:46.633809+0000","last_clean":"2023-10-16T08:38:46.633809+0000","last_became_active":"2023-10-16T08:38:45.665972+0000","last_became_peered":"2023-10-16T08:38:45.665972+0000","last_unstale":"2023-10-16T08:38:46.633809+0000","last_undegraded":"2023-10-16T08:38:46.633809+0000","last_fullsized":"2023-10-16T08:38:46.633809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626436+0000","last_change":"2023-10-16T08:38:45.667047+0000","last_active":"2023-10-16T08:38:46.626436+0000","last_peered":"2023-10-16T08:38:46.626436+0000","last_clean":"2023-10-16T08:38:46.626436+0000","last_became_active":"2023-10-16T08:38:45.666403+0000","last_became_peered":"2023-10-16T08:38:45.666403+0000","last_unstale":"2023-10-16T08:38:46.626436+0000","last_undegraded":"2023-10-16T08:38:46.626436+0000","last_fullsized":"2023-10-16T08:38:46.626436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628642+0000","last_change":"2023-10-16T08:38:45.668007+0000","last_active":"2023-10-16T08:38:46.628642+0000","last_peered":"2023-10-16T08:38:46.628642+0000","last_clean":"2023-10-16T08:38:46.628642+0000","last_became_active":"2023-10-16T08:38:45.664572+0000","last_became_peered":"2023-10-16T08:38:45.664572+0000","last_unstale":"2023-10-16T08:38:46.628642+0000","last_undegraded":"2023-10-16T08:38:46.628642+0000","last_fullsized":"2023-10-16T08:38:46.628642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926222+0000","last_change":"2023-10-16T08:38:45.667378+0000","last_active":"2023-10-16T08:38:46.926222+0000","last_peered":"2023-10-16T08:38:46.926222+0000","last_clean":"2023-10-16T08:38:46.926222+0000","last_became_active":"2023-10-16T08:38:45.666547+0000","last_became_peered":"2023-10-16T08:38:45.666547+0000","last_unstale":"2023-10-16T08:38:46.926222+0000","last_undegraded":"2023-10-16T08:38:46.926222+0000","last_fullsized":"2023-10-16T08:38:46.926222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931201+0000","last_change":"2023-10-16T08:38:45.669071+0000","last_active":"2023-10-16T08:38:46.931201+0000","last_peered":"2023-10-16T08:38:46.931201+0000","last_clean":"2023-10-16T08:38:46.931201+0000","last_became_active":"2023-10-16T08:38:45.666591+0000","last_became_peered":"2023-10-16T08:38:45.666591+0000","last_unstale":"2023-10-16T08:38:46.931201+0000","last_undegraded":"2023-10-16T08:38:46.931201+0000","last_fullsized":"2023-10-16T08:38:46.931201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:02:39.840175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927293+0000","last_change":"2023-10-16T08:38:45.648288+0000","last_active":"2023-10-16T08:38:46.927293+0000","last_peered":"2023-10-16T08:38:46.927293+0000","last_clean":"2023-10-16T08:38:46.927293+0000","last_became_active":"2023-10-16T08:38:45.647484+0000","last_became_peered":"2023-10-16T08:38:45.647484+0000","last_unstale":"2023-10-16T08:38:46.927293+0000","last_undegraded":"2023-10-16T08:38:46.927293+0000","last_fullsized":"2023-10-16T08:38:46.927293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:20.779617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.632427+0000","last_change":"2023-10-16T08:38:45.638183+0000","last_active":"2023-10-16T08:38:46.632427+0000","last_peered":"2023-10-16T08:38:46.632427+0000","last_clean":"2023-10-16T08:38:46.632427+0000","last_became_active":"2023-10-16T08:38:45.636762+0000","last_became_peered":"2023-10-16T08:38:45.636762+0000","last_unstale":"2023-10-16T08:38:46.632427+0000","last_undegraded":"2023-10-16T08:38:46.632427+0000","last_fullsized":"2023-10-16T08:38:46.632427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.929995+0000","last_change":"2023-10-16T08:38:45.659464+0000","last_active":"2023-10-16T08:38:46.929995+0000","last_peered":"2023-10-16T08:38:46.929995+0000","last_clean":"2023-10-16T08:38:46.929995+0000","last_became_active":"2023-10-16T08:38:45.658808+0000","last_became_peered":"2023-10-16T08:38:45.658808+0000","last_unstale":"2023-10-16T08:38:46.929995+0000","last_undegraded":"2023-10-16T08:38:46.929995+0000","last_fullsized":"2023-10-16T08:38:46.929995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:25.386221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628419+0000","last_change":"2023-10-16T08:38:44.644122+0000","last_active":"2023-10-16T08:38:46.628419+0000","last_peered":"2023-10-16T08:38:46.628419+0000","last_clean":"2023-10-16T08:38:46.628419+0000","last_became_active":"2023-10-16T08:38:44.643614+0000","last_became_peered":"2023-10-16T08:38:44.643614+0000","last_unstale":"2023-10-16T08:38:46.628419+0000","last_undegraded":"2023-10-16T08:38:46.628419+0000","last_fullsized":"2023-10-16T08:38:46.628419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926570+0000","last_change":"2023-10-16T08:38:45.659854+0000","last_active":"2023-10-16T08:38:46.926570+0000","last_peered":"2023-10-16T08:38:46.926570+0000","last_clean":"2023-10-16T08:38:46.926570+0000","last_became_active":"2023-10-16T08:38:45.658409+0000","last_became_peered":"2023-10-16T08:38:45.658409+0000","last_unstale":"2023-10-16T08:38:46.926570+0000","last_undegraded":"2023-10-16T08:38:46.926570+0000","last_fullsized":"2023-10-16T08:38:46.926570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.632862+0000","last_change":"2023-10-16T08:38:44.658819+0000","last_active":"2023-10-16T08:38:46.632862+0000","last_peered":"2023-10-16T08:38:46.632862+0000","last_clean":"2023-10-16T08:38:46.632862+0000","last_became_active":"2023-10-16T08:38:44.658257+0000","last_became_peered":"2023-10-16T08:38:44.658257+0000","last_unstale":"2023-10-16T08:38:46.632862+0000","last_undegraded":"2023-10-16T08:38:46.632862+0000","last_fullsized":"2023-10-16T08:38:46.632862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:54.848445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628283+0000","last_change":"2023-10-16T08:38:44.647536+0000","last_active":"2023-10-16T08:38:46.628283+0000","last_peered":"2023-10-16T08:38:46.628283+0000","last_clean":"2023-10-16T08:38:46.628283+0000","last_became_active":"2023-10-16T08:38:44.645931+0000","last_became_peered":"2023-10-16T08:38:44.645931+0000","last_unstale":"2023-10-16T08:38:46.628283+0000","last_undegraded":"2023-10-16T08:38:46.628283+0000","last_fullsized":"2023-10-16T08:38:46.628283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628223+0000","last_change":"2023-10-16T08:38:45.662855+0000","last_active":"2023-10-16T08:38:46.628223+0000","last_peered":"2023-10-16T08:38:46.628223+0000","last_clean":"2023-10-16T08:38:46.628223+0000","last_became_active":"2023-10-16T08:38:45.662072+0000","last_became_peered":"2023-10-16T08:38:45.662072+0000","last_unstale":"2023-10-16T08:38:46.628223+0000","last_undegraded":"2023-10-16T08:38:46.628223+0000","last_fullsized":"2023-10-16T08:38:46.628223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.929953+0000","last_change":"2023-10-16T08:38:44.657651+0000","last_active":"2023-10-16T08:38:46.929953+0000","last_peered":"2023-10-16T08:38:46.929953+0000","last_clean":"2023-10-16T08:38:46.929953+0000","last_became_active":"2023-10-16T08:38:44.657065+0000","last_became_peered":"2023-10-16T08:38:44.657065+0000","last_unstale":"2023-10-16T08:38:46.929953+0000","last_undegraded":"2023-10-16T08:38:46.929953+0000","last_fullsized":"2023-10-16T08:38:46.929953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.625975+0000","last_change":"2023-10-16T08:38:45.668048+0000","last_active":"2023-10-16T08:38:46.625975+0000","last_peered":"2023-10-16T08:38:46.625975+0000","last_clean":"2023-10-16T08:38:46.625975+0000","last_became_active":"2023-10-16T08:38:45.667413+0000","last_became_peered":"2023-10-16T08:38:45.667413+0000","last_unstale":"2023-10-16T08:38:46.625975+0000","last_undegraded":"2023-10-16T08:38:46.625975+0000","last_fullsized":"2023-10-16T08:38:46.625975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633561+0000","last_change":"2023-10-16T08:38:44.663188+0000","last_active":"2023-10-16T08:38:46.633561+0000","last_peered":"2023-10-16T08:38:46.633561+0000","last_clean":"2023-10-16T08:38:46.633561+0000","last_became_active":"2023-10-16T08:38:44.662383+0000","last_became_peered":"2023-10-16T08:38:44.662383+0000","last_unstale":"2023-10-16T08:38:46.633561+0000","last_undegraded":"2023-10-16T08:38:46.633561+0000","last_fullsized":"2023-10-16T08:38:46.633561+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:24.375366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930913+0000","last_change":"2023-10-16T08:38:45.679271+0000","last_active":"2023-10-16T08:38:46.930913+0000","last_peered":"2023-10-16T08:38:46.930913+0000","last_clean":"2023-10-16T08:38:46.930913+0000","last_became_active":"2023-10-16T08:38:45.678050+0000","last_became_peered":"2023-10-16T08:38:45.678050+0000","last_unstale":"2023-10-16T08:38:46.930913+0000","last_undegraded":"2023-10-16T08:38:46.930913+0000","last_fullsized":"2023-10-16T08:38:46.930913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931150+0000","last_change":"2023-10-16T08:38:44.653478+0000","last_active":"2023-10-16T08:38:46.931150+0000","last_peered":"2023-10-16T08:38:46.931150+0000","last_clean":"2023-10-16T08:38:46.931150+0000","last_became_active":"2023-10-16T08:38:44.652618+0000","last_became_peered":"2023-10-16T08:38:44.652618+0000","last_unstale":"2023-10-16T08:38:46.931150+0000","last_undegraded":"2023-10-16T08:38:46.931150+0000","last_fullsized":"2023-10-16T08:38:46.931150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926242+0000","last_change":"2023-10-16T08:38:45.656603+0000","last_active":"2023-10-16T08:38:46.926242+0000","last_peered":"2023-10-16T08:38:46.926242+0000","last_clean":"2023-10-16T08:38:46.926242+0000","last_became_active":"2023-10-16T08:38:45.655213+0000","last_became_peered":"2023-10-16T08:38:45.655213+0000","last_unstale":"2023-10-16T08:38:46.926242+0000","last_undegraded":"2023-10-16T08:38:46.926242+0000","last_fullsized":"2023-10-16T08:38:46.926242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626622+0000","last_change":"2023-10-16T08:38:44.651637+0000","last_active":"2023-10-16T08:38:46.626622+0000","last_peered":"2023-10-16T08:38:46.626622+0000","last_clean":"2023-10-16T08:38:46.626622+0000","last_became_active":"2023-10-16T08:38:44.651359+0000","last_became_peered":"2023-10-16T08:38:44.651359+0000","last_unstale":"2023-10-16T08:38:46.626622+0000","last_undegraded":"2023-10-16T08:38:46.626622+0000","last_fullsized":"2023-10-16T08:38:46.626622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:10.859316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.932633+0000","last_change":"2023-10-16T08:38:45.661741+0000","last_active":"2023-10-16T08:38:46.932633+0000","last_peered":"2023-10-16T08:38:46.932633+0000","last_clean":"2023-10-16T08:38:46.932633+0000","last_became_active":"2023-10-16T08:38:45.660611+0000","last_became_peered":"2023-10-16T08:38:45.660611+0000","last_unstale":"2023-10-16T08:38:46.932633+0000","last_undegraded":"2023-10-16T08:38:46.932633+0000","last_fullsized":"2023-10-16T08:38:46.932633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:55.859300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633359+0000","last_change":"2023-10-16T08:38:44.647593+0000","last_active":"2023-10-16T08:38:46.633359+0000","last_peered":"2023-10-16T08:38:46.633359+0000","last_clean":"2023-10-16T08:38:46.633359+0000","last_became_active":"2023-10-16T08:38:44.647001+0000","last_became_peered":"2023-10-16T08:38:44.647001+0000","last_unstale":"2023-10-16T08:38:46.633359+0000","last_undegraded":"2023-10-16T08:38:46.633359+0000","last_fullsized":"2023-10-16T08:38:46.633359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930291+0000","last_change":"2023-10-16T08:38:45.668629+0000","last_active":"2023-10-16T08:38:46.930291+0000","last_peered":"2023-10-16T08:38:46.930291+0000","last_clean":"2023-10-16T08:38:46.930291+0000","last_became_active":"2023-10-16T08:38:45.667334+0000","last_became_peered":"2023-10-16T08:38:45.667334+0000","last_unstale":"2023-10-16T08:38:46.930291+0000","last_undegraded":"2023-10-16T08:38:46.930291+0000","last_fullsized":"2023-10-16T08:38:46.930291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926736+0000","last_change":"2023-10-16T08:38:44.655832+0000","last_active":"2023-10-16T08:38:46.926736+0000","last_peered":"2023-10-16T08:38:46.926736+0000","last_clean":"2023-10-16T08:38:46.926736+0000","last_became_active":"2023-10-16T08:38:44.655469+0000","last_became_peered":"2023-10-16T08:38:44.655469+0000","last_unstale":"2023-10-16T08:38:46.926736+0000","last_undegraded":"2023-10-16T08:38:46.926736+0000","last_fullsized":"2023-10-16T08:38:46.926736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:19.768762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926112+0000","last_change":"2023-10-16T08:38:45.653688+0000","last_active":"2023-10-16T08:38:46.926112+0000","last_peered":"2023-10-16T08:38:46.926112+0000","last_clean":"2023-10-16T08:38:46.926112+0000","last_became_active":"2023-10-16T08:38:45.650563+0000","last_became_peered":"2023-10-16T08:38:45.650563+0000","last_unstale":"2023-10-16T08:38:46.926112+0000","last_undegraded":"2023-10-16T08:38:46.926112+0000","last_fullsized":"2023-10-16T08:38:46.926112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:29.539285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931330+0000","last_change":"2023-10-16T08:38:44.662816+0000","last_active":"2023-10-16T08:38:46.931330+0000","last_peered":"2023-10-16T08:38:46.931330+0000","last_clean":"2023-10-16T08:38:46.931330+0000","last_became_active":"2023-10-16T08:38:44.662110+0000","last_became_peered":"2023-10-16T08:38:44.662110+0000","last_unstale":"2023-10-16T08:38:46.931330+0000","last_undegraded":"2023-10-16T08:38:46.931330+0000","last_fullsized":"2023-10-16T08:38:46.931330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:00.201724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930788+0000","last_change":"2023-10-16T08:38:45.662166+0000","last_active":"2023-10-16T08:38:46.930788+0000","last_peered":"2023-10-16T08:38:46.930788+0000","last_clean":"2023-10-16T08:38:46.930788+0000","last_became_active":"2023-10-16T08:38:45.661595+0000","last_became_peered":"2023-10-16T08:38:45.661595+0000","last_unstale":"2023-10-16T08:38:46.930788+0000","last_undegraded":"2023-10-16T08:38:46.930788+0000","last_fullsized":"2023-10-16T08:38:46.930788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:02:39.840175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.632731+0000","last_change":"2023-10-16T08:38:44.657766+0000","last_active":"2023-10-16T08:38:46.632731+0000","last_peered":"2023-10-16T08:38:46.632731+0000","last_clean":"2023-10-16T08:38:46.632731+0000","last_became_active":"2023-10-16T08:38:44.657153+0000","last_became_peered":"2023-10-16T08:38:44.657153+0000","last_unstale":"2023-10-16T08:38:46.632731+0000","last_undegraded":"2023-10-16T08:38:46.632731+0000","last_fullsized":"2023-10-16T08:38:46.632731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.632435+0000","last_change":"2023-10-16T08:38:45.637591+0000","last_active":"2023-10-16T08:38:46.632435+0000","last_peered":"2023-10-16T08:38:46.632435+0000","last_clean":"2023-10-16T08:38:46.632435+0000","last_became_active":"2023-10-16T08:38:45.635633+0000","last_became_peered":"2023-10-16T08:38:45.635633+0000","last_unstale":"2023-10-16T08:38:46.632435+0000","last_undegraded":"2023-10-16T08:38:46.632435+0000","last_fullsized":"2023-10-16T08:38:46.632435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927363+0000","last_change":"2023-10-16T08:38:44.638297+0000","last_active":"2023-10-16T08:38:46.927363+0000","last_peered":"2023-10-16T08:38:46.927363+0000","last_clean":"2023-10-16T08:38:46.927363+0000","last_became_active":"2023-10-16T08:38:44.637640+0000","last_became_peered":"2023-10-16T08:38:44.637640+0000","last_unstale":"2023-10-16T08:38:46.927363+0000","last_undegraded":"2023-10-16T08:38:46.927363+0000","last_fullsized":"2023-10-16T08:38:46.927363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:16.003943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931498+0000","last_change":"2023-10-16T08:38:45.657286+0000","last_active":"2023-10-16T08:38:46.931498+0000","last_peered":"2023-10-16T08:38:46.931498+0000","last_clean":"2023-10-16T08:38:46.931498+0000","last_became_active":"2023-10-16T08:38:45.656651+0000","last_became_peered":"2023-10-16T08:38:45.656651+0000","last_unstale":"2023-10-16T08:38:46.931498+0000","last_undegraded":"2023-10-16T08:38:46.931498+0000","last_fullsized":"2023-10-16T08:38:46.931498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:28.052483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.932737+0000","last_change":"2023-10-16T08:38:44.648434+0000","last_active":"2023-10-16T08:38:46.932737+0000","last_peered":"2023-10-16T08:38:46.932737+0000","last_clean":"2023-10-16T08:38:46.932737+0000","last_became_active":"2023-10-16T08:38:44.647887+0000","last_became_peered":"2023-10-16T08:38:44.647887+0000","last_unstale":"2023-10-16T08:38:46.932737+0000","last_undegraded":"2023-10-16T08:38:46.932737+0000","last_fullsized":"2023-10-16T08:38:46.932737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930414+0000","last_change":"2023-10-16T08:38:45.661965+0000","last_active":"2023-10-16T08:38:46.930414+0000","last_peered":"2023-10-16T08:38:46.930414+0000","last_clean":"2023-10-16T08:38:46.930414+0000","last_became_active":"2023-10-16T08:38:45.656750+0000","last_became_peered":"2023-10-16T08:38:45.656750+0000","last_unstale":"2023-10-16T08:38:46.930414+0000","last_undegraded":"2023-10-16T08:38:46.930414+0000","last_fullsized":"2023-10-16T08:38:46.930414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927746+0000","last_change":"2023-10-16T08:38:44.647524+0000","last_active":"2023-10-16T08:38:46.927746+0000","last_peered":"2023-10-16T08:38:46.927746+0000","last_clean":"2023-10-16T08:38:46.927746+0000","last_became_active":"2023-10-16T08:38:44.646042+0000","last_became_peered":"2023-10-16T08:38:44.646042+0000","last_unstale":"2023-10-16T08:38:46.927746+0000","last_undegraded":"2023-10-16T08:38:46.927746+0000","last_fullsized":"2023-10-16T08:38:46.927746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:24.375366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931909+0000","last_change":"2023-10-16T08:38:45.670969+0000","last_active":"2023-10-16T08:38:46.931909+0000","last_peered":"2023-10-16T08:38:46.931909+0000","last_clean":"2023-10-16T08:38:46.931909+0000","last_became_active":"2023-10-16T08:38:45.670415+0000","last_became_peered":"2023-10-16T08:38:45.670415+0000","last_unstale":"2023-10-16T08:38:46.931909+0000","last_undegraded":"2023-10-16T08:38:46.931909+0000","last_fullsized":"2023-10-16T08:38:46.931909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:15:00.472153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626608+0000","last_change":"2023-10-16T08:38:44.655744+0000","last_active":"2023-10-16T08:38:46.626608+0000","last_peered":"2023-10-16T08:38:46.626608+0000","last_clean":"2023-10-16T08:38:46.626608+0000","last_became_active":"2023-10-16T08:38:44.655260+0000","last_became_peered":"2023-10-16T08:38:44.655260+0000","last_unstale":"2023-10-16T08:38:46.626608+0000","last_undegraded":"2023-10-16T08:38:46.626608+0000","last_fullsized":"2023-10-16T08:38:46.626608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:54.848445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931710+0000","last_change":"2023-10-16T08:38:45.668230+0000","last_active":"2023-10-16T08:38:46.931710+0000","last_peered":"2023-10-16T08:38:46.931710+0000","last_clean":"2023-10-16T08:38:46.931710+0000","last_became_active":"2023-10-16T08:38:45.667627+0000","last_became_peered":"2023-10-16T08:38:45.667627+0000","last_unstale":"2023-10-16T08:38:46.931710+0000","last_undegraded":"2023-10-16T08:38:46.931710+0000","last_fullsized":"2023-10-16T08:38:46.931710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:28.052483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927508+0000","last_change":"2023-10-16T08:38:44.662118+0000","last_active":"2023-10-16T08:38:46.927508+0000","last_peered":"2023-10-16T08:38:46.927508+0000","last_clean":"2023-10-16T08:38:46.927508+0000","last_became_active":"2023-10-16T08:38:44.661736+0000","last_became_peered":"2023-10-16T08:38:44.661736+0000","last_unstale":"2023-10-16T08:38:46.927508+0000","last_undegraded":"2023-10-16T08:38:46.927508+0000","last_fullsized":"2023-10-16T08:38:46.927508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:49:40.967349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930619+0000","last_change":"2023-10-16T08:38:45.667423+0000","last_active":"2023-10-16T08:38:46.930619+0000","last_peered":"2023-10-16T08:38:46.930619+0000","last_clean":"2023-10-16T08:38:46.930619+0000","last_became_active":"2023-10-16T08:38:45.666390+0000","last_became_peered":"2023-10-16T08:38:45.666390+0000","last_unstale":"2023-10-16T08:38:46.930619+0000","last_undegraded":"2023-10-16T08:38:46.930619+0000","last_fullsized":"2023-10-16T08:38:46.930619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:33.190218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-16T08:38:45.628438+0000","last_change":"2023-10-16T08:38:34.313511+0000","last_active":"2023-10-16T08:38:45.628438+0000","last_peered":"2023-10-16T08:38:45.628438+0000","last_clean":"2023-10-16T08:38:45.628438+0000","last_became_active":"2023-10-16T08:38:34.310944+0000","last_became_peered":"2023-10-16T08:38:34.310944+0000","last_unstale":"2023-10-16T08:38:45.628438+0000","last_undegraded":"2023-10-16T08:38:45.628438+0000","last_fullsized":"2023-10-16T08:38:45.628438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:33.264692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:33.264692+0000","last_clean_scrub_stamp":"2023-10-16T08:38:33.264692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:50.170461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930271+0000","last_change":"2023-10-16T08:38:44.655223+0000","last_active":"2023-10-16T08:38:46.930271+0000","last_peered":"2023-10-16T08:38:46.930271+0000","last_clean":"2023-10-16T08:38:46.930271+0000","last_became_active":"2023-10-16T08:38:44.654277+0000","last_became_peered":"2023-10-16T08:38:44.654277+0000","last_unstale":"2023-10-16T08:38:46.930271+0000","last_undegraded":"2023-10-16T08:38:46.930271+0000","last_fullsized":"2023-10-16T08:38:46.930271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:03.462433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628392+0000","last_change":"2023-10-16T08:38:45.679868+0000","last_active":"2023-10-16T08:38:46.628392+0000","last_peered":"2023-10-16T08:38:46.628392+0000","last_clean":"2023-10-16T08:38:46.628392+0000","last_became_active":"2023-10-16T08:38:45.679409+0000","last_became_peered":"2023-10-16T08:38:45.679409+0000","last_unstale":"2023-10-16T08:38:46.628392+0000","last_undegraded":"2023-10-16T08:38:46.628392+0000","last_fullsized":"2023-10-16T08:38:46.628392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930490+0000","last_change":"2023-10-16T08:38:44.652152+0000","last_active":"2023-10-16T08:38:46.930490+0000","last_peered":"2023-10-16T08:38:46.930490+0000","last_clean":"2023-10-16T08:38:46.930490+0000","last_became_active":"2023-10-16T08:38:44.651534+0000","last_became_peered":"2023-10-16T08:38:44.651534+0000","last_unstale":"2023-10-16T08:38:46.930490+0000","last_undegraded":"2023-10-16T08:38:46.930490+0000","last_fullsized":"2023-10-16T08:38:46.930490+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926115+0000","last_change":"2023-10-16T08:38:45.651786+0000","last_active":"2023-10-16T08:38:46.926115+0000","last_peered":"2023-10-16T08:38:46.926115+0000","last_clean":"2023-10-16T08:38:46.926115+0000","last_became_active":"2023-10-16T08:38:45.650715+0000","last_became_peered":"2023-10-16T08:38:45.650715+0000","last_unstale":"2023-10-16T08:38:46.926115+0000","last_undegraded":"2023-10-16T08:38:46.926115+0000","last_fullsized":"2023-10-16T08:38:46.926115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:29.799799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927326+0000","last_change":"2023-10-16T08:38:44.645933+0000","last_active":"2023-10-16T08:38:46.927326+0000","last_peered":"2023-10-16T08:38:46.927326+0000","last_clean":"2023-10-16T08:38:46.927326+0000","last_became_active":"2023-10-16T08:38:44.645451+0000","last_became_peered":"2023-10-16T08:38:44.645451+0000","last_unstale":"2023-10-16T08:38:46.927326+0000","last_undegraded":"2023-10-16T08:38:46.927326+0000","last_fullsized":"2023-10-16T08:38:46.927326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.625977+0000","last_change":"2023-10-16T08:38:45.641499+0000","last_active":"2023-10-16T08:38:46.625977+0000","last_peered":"2023-10-16T08:38:46.625977+0000","last_clean":"2023-10-16T08:38:46.625977+0000","last_became_active":"2023-10-16T08:38:45.640765+0000","last_became_peered":"2023-10-16T08:38:45.640765+0000","last_unstale":"2023-10-16T08:38:46.625977+0000","last_undegraded":"2023-10-16T08:38:46.625977+0000","last_fullsized":"2023-10-16T08:38:46.625977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931417+0000","last_change":"2023-10-16T08:38:44.655401+0000","last_active":"2023-10-16T08:38:46.931417+0000","last_peered":"2023-10-16T08:38:46.931417+0000","last_clean":"2023-10-16T08:38:46.931417+0000","last_became_active":"2023-10-16T08:38:44.654622+0000","last_became_peered":"2023-10-16T08:38:44.654622+0000","last_unstale":"2023-10-16T08:38:46.931417+0000","last_undegraded":"2023-10-16T08:38:46.931417+0000","last_fullsized":"2023-10-16T08:38:46.931417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:48.705915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.932454+0000","last_change":"2023-10-16T08:38:45.659377+0000","last_active":"2023-10-16T08:38:46.932454+0000","last_peered":"2023-10-16T08:38:46.932454+0000","last_clean":"2023-10-16T08:38:46.932454+0000","last_became_active":"2023-10-16T08:38:45.658594+0000","last_became_peered":"2023-10-16T08:38:45.658594+0000","last_unstale":"2023-10-16T08:38:46.932454+0000","last_undegraded":"2023-10-16T08:38:46.932454+0000","last_fullsized":"2023-10-16T08:38:46.932454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:56.738058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626155+0000","last_change":"2023-10-16T08:38:44.634158+0000","last_active":"2023-10-16T08:38:46.626155+0000","last_peered":"2023-10-16T08:38:46.626155+0000","last_clean":"2023-10-16T08:38:46.626155+0000","last_became_active":"2023-10-16T08:38:44.633854+0000","last_became_peered":"2023-10-16T08:38:44.633854+0000","last_unstale":"2023-10-16T08:38:46.626155+0000","last_undegraded":"2023-10-16T08:38:46.626155+0000","last_fullsized":"2023-10-16T08:38:46.626155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:55.727203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931328+0000","last_change":"2023-10-16T08:38:45.675931+0000","last_active":"2023-10-16T08:38:46.931328+0000","last_peered":"2023-10-16T08:38:46.931328+0000","last_clean":"2023-10-16T08:38:46.931328+0000","last_became_active":"2023-10-16T08:38:45.674962+0000","last_became_peered":"2023-10-16T08:38:45.674962+0000","last_unstale":"2023-10-16T08:38:46.931328+0000","last_undegraded":"2023-10-16T08:38:46.931328+0000","last_fullsized":"2023-10-16T08:38:46.931328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:13.453321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931397+0000","last_change":"2023-10-16T08:38:44.655693+0000","last_active":"2023-10-16T08:38:46.931397+0000","last_peered":"2023-10-16T08:38:46.931397+0000","last_clean":"2023-10-16T08:38:46.931397+0000","last_became_active":"2023-10-16T08:38:44.655001+0000","last_became_peered":"2023-10-16T08:38:44.655001+0000","last_unstale":"2023-10-16T08:38:46.931397+0000","last_undegraded":"2023-10-16T08:38:46.931397+0000","last_fullsized":"2023-10-16T08:38:46.931397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:30.649639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626084+0000","last_change":"2023-10-16T08:38:45.639658+0000","last_active":"2023-10-16T08:38:46.626084+0000","last_peered":"2023-10-16T08:38:46.626084+0000","last_clean":"2023-10-16T08:38:46.626084+0000","last_became_active":"2023-10-16T08:38:45.638702+0000","last_became_peered":"2023-10-16T08:38:45.638702+0000","last_unstale":"2023-10-16T08:38:46.626084+0000","last_undegraded":"2023-10-16T08:38:46.626084+0000","last_fullsized":"2023-10-16T08:38:46.626084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626687+0000","last_change":"2023-10-16T08:38:44.648715+0000","last_active":"2023-10-16T08:38:46.626687+0000","last_peered":"2023-10-16T08:38:46.626687+0000","last_clean":"2023-10-16T08:38:46.626687+0000","last_became_active":"2023-10-16T08:38:44.648312+0000","last_became_peered":"2023-10-16T08:38:44.648312+0000","last_unstale":"2023-10-16T08:38:46.626687+0000","last_undegraded":"2023-10-16T08:38:46.626687+0000","last_fullsized":"2023-10-16T08:38:46.626687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628807+0000","last_change":"2023-10-16T08:38:45.663402+0000","last_active":"2023-10-16T08:38:46.628807+0000","last_peered":"2023-10-16T08:38:46.628807+0000","last_clean":"2023-10-16T08:38:46.628807+0000","last_became_active":"2023-10-16T08:38:45.662442+0000","last_became_peered":"2023-10-16T08:38:45.662442+0000","last_unstale":"2023-10-16T08:38:46.628807+0000","last_undegraded":"2023-10-16T08:38:46.628807+0000","last_fullsized":"2023-10-16T08:38:46.628807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633639+0000","last_change":"2023-10-16T08:38:44.650015+0000","last_active":"2023-10-16T08:38:46.633639+0000","last_peered":"2023-10-16T08:38:46.633639+0000","last_clean":"2023-10-16T08:38:46.633639+0000","last_became_active":"2023-10-16T08:38:44.648155+0000","last_became_peered":"2023-10-16T08:38:44.648155+0000","last_unstale":"2023-10-16T08:38:46.633639+0000","last_undegraded":"2023-10-16T08:38:46.633639+0000","last_fullsized":"2023-10-16T08:38:46.633639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926726+0000","last_change":"2023-10-16T08:38:45.637568+0000","last_active":"2023-10-16T08:38:46.926726+0000","last_peered":"2023-10-16T08:38:46.926726+0000","last_clean":"2023-10-16T08:38:46.926726+0000","last_became_active":"2023-10-16T08:38:45.636751+0000","last_became_peered":"2023-10-16T08:38:45.636751+0000","last_unstale":"2023-10-16T08:38:46.926726+0000","last_undegraded":"2023-10-16T08:38:46.926726+0000","last_fullsized":"2023-10-16T08:38:46.926726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930247+0000","last_change":"2023-10-16T08:38:44.649257+0000","last_active":"2023-10-16T08:38:46.930247+0000","last_peered":"2023-10-16T08:38:46.930247+0000","last_clean":"2023-10-16T08:38:46.930247+0000","last_became_active":"2023-10-16T08:38:44.646990+0000","last_became_peered":"2023-10-16T08:38:44.646990+0000","last_unstale":"2023-10-16T08:38:46.930247+0000","last_undegraded":"2023-10-16T08:38:46.930247+0000","last_fullsized":"2023-10-16T08:38:46.930247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930389+0000","last_change":"2023-10-16T08:38:45.648228+0000","last_active":"2023-10-16T08:38:46.930389+0000","last_peered":"2023-10-16T08:38:46.930389+0000","last_clean":"2023-10-16T08:38:46.930389+0000","last_became_active":"2023-10-16T08:38:45.647449+0000","last_became_peered":"2023-10-16T08:38:45.647449+0000","last_unstale":"2023-10-16T08:38:46.930389+0000","last_undegraded":"2023-10-16T08:38:46.930389+0000","last_fullsized":"2023-10-16T08:38:46.930389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:20.779617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626045+0000","last_change":"2023-10-16T08:38:44.631354+0000","last_active":"2023-10-16T08:38:46.626045+0000","last_peered":"2023-10-16T08:38:46.626045+0000","last_clean":"2023-10-16T08:38:46.626045+0000","last_became_active":"2023-10-16T08:38:44.630451+0000","last_became_peered":"2023-10-16T08:38:44.630451+0000","last_unstale":"2023-10-16T08:38:46.626045+0000","last_undegraded":"2023-10-16T08:38:46.626045+0000","last_fullsized":"2023-10-16T08:38:46.626045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:10.980982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628323+0000","last_change":"2023-10-16T08:38:45.663898+0000","last_active":"2023-10-16T08:38:46.628323+0000","last_peered":"2023-10-16T08:38:46.628323+0000","last_clean":"2023-10-16T08:38:46.628323+0000","last_became_active":"2023-10-16T08:38:45.663100+0000","last_became_peered":"2023-10-16T08:38:45.663100+0000","last_unstale":"2023-10-16T08:38:46.628323+0000","last_undegraded":"2023-10-16T08:38:46.628323+0000","last_fullsized":"2023-10-16T08:38:46.628323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626257+0000","last_change":"2023-10-16T08:38:44.646987+0000","last_active":"2023-10-16T08:38:46.626257+0000","last_peered":"2023-10-16T08:38:46.626257+0000","last_clean":"2023-10-16T08:38:46.626257+0000","last_became_active":"2023-10-16T08:38:44.646580+0000","last_became_peered":"2023-10-16T08:38:44.646580+0000","last_unstale":"2023-10-16T08:38:46.626257+0000","last_undegraded":"2023-10-16T08:38:46.626257+0000","last_fullsized":"2023-10-16T08:38:46.626257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628448+0000","last_change":"2023-10-16T08:38:45.662894+0000","last_active":"2023-10-16T08:38:46.628448+0000","last_peered":"2023-10-16T08:38:46.628448+0000","last_clean":"2023-10-16T08:38:46.628448+0000","last_became_active":"2023-10-16T08:38:45.662226+0000","last_became_peered":"2023-10-16T08:38:45.662226+0000","last_unstale":"2023-10-16T08:38:46.628448+0000","last_undegraded":"2023-10-16T08:38:46.628448+0000","last_fullsized":"2023-10-16T08:38:46.628448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931043+0000","last_change":"2023-10-16T08:38:44.660571+0000","last_active":"2023-10-16T08:38:46.931043+0000","last_peered":"2023-10-16T08:38:46.931043+0000","last_clean":"2023-10-16T08:38:46.931043+0000","last_became_active":"2023-10-16T08:38:44.659976+0000","last_became_peered":"2023-10-16T08:38:44.659976+0000","last_unstale":"2023-10-16T08:38:46.931043+0000","last_undegraded":"2023-10-16T08:38:46.931043+0000","last_fullsized":"2023-10-16T08:38:46.931043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.632966+0000","last_change":"2023-10-16T08:38:45.665200+0000","last_active":"2023-10-16T08:38:46.632966+0000","last_peered":"2023-10-16T08:38:46.632966+0000","last_clean":"2023-10-16T08:38:46.632966+0000","last_became_active":"2023-10-16T08:38:45.664616+0000","last_became_peered":"2023-10-16T08:38:45.664616+0000","last_unstale":"2023-10-16T08:38:46.632966+0000","last_undegraded":"2023-10-16T08:38:46.632966+0000","last_fullsized":"2023-10-16T08:38:46.632966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.933149+0000","last_change":"2023-10-16T08:38:45.680892+0000","last_active":"2023-10-16T08:38:46.933149+0000","last_peered":"2023-10-16T08:38:46.933149+0000","last_clean":"2023-10-16T08:38:46.933149+0000","last_became_active":"2023-10-16T08:38:45.680231+0000","last_became_peered":"2023-10-16T08:38:45.680231+0000","last_unstale":"2023-10-16T08:38:46.933149+0000","last_undegraded":"2023-10-16T08:38:46.933149+0000","last_fullsized":"2023-10-16T08:38:46.933149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:58:51.599709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.929904+0000","last_change":"2023-10-16T08:38:44.664269+0000","last_active":"2023-10-16T08:38:46.929904+0000","last_peered":"2023-10-16T08:38:46.929904+0000","last_clean":"2023-10-16T08:38:46.929904+0000","last_became_active":"2023-10-16T08:38:44.663674+0000","last_became_peered":"2023-10-16T08:38:44.663674+0000","last_unstale":"2023-10-16T08:38:46.929904+0000","last_undegraded":"2023-10-16T08:38:46.929904+0000","last_fullsized":"2023-10-16T08:38:46.929904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931372+0000","last_change":"2023-10-16T08:38:45.680890+0000","last_active":"2023-10-16T08:38:46.931372+0000","last_peered":"2023-10-16T08:38:46.931372+0000","last_clean":"2023-10-16T08:38:46.931372+0000","last_became_active":"2023-10-16T08:38:45.680266+0000","last_became_peered":"2023-10-16T08:38:45.680266+0000","last_unstale":"2023-10-16T08:38:46.931372+0000","last_undegraded":"2023-10-16T08:38:46.931372+0000","last_fullsized":"2023-10-16T08:38:46.931372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:26.292137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930850+0000","last_change":"2023-10-16T08:38:44.653447+0000","last_active":"2023-10-16T08:38:46.930850+0000","last_peered":"2023-10-16T08:38:46.930850+0000","last_clean":"2023-10-16T08:38:46.930850+0000","last_became_active":"2023-10-16T08:38:44.652658+0000","last_became_peered":"2023-10-16T08:38:44.652658+0000","last_unstale":"2023-10-16T08:38:46.930850+0000","last_undegraded":"2023-10-16T08:38:46.930850+0000","last_fullsized":"2023-10-16T08:38:46.930850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:22.350066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926151+0000","last_change":"2023-10-16T08:38:45.665041+0000","last_active":"2023-10-16T08:38:46.926151+0000","last_peered":"2023-10-16T08:38:46.926151+0000","last_clean":"2023-10-16T08:38:46.926151+0000","last_became_active":"2023-10-16T08:38:45.664274+0000","last_became_peered":"2023-10-16T08:38:45.664274+0000","last_unstale":"2023-10-16T08:38:46.926151+0000","last_undegraded":"2023-10-16T08:38:46.926151+0000","last_fullsized":"2023-10-16T08:38:46.926151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628731+0000","last_change":"2023-10-16T08:38:44.645754+0000","last_active":"2023-10-16T08:38:46.628731+0000","last_peered":"2023-10-16T08:38:46.628731+0000","last_clean":"2023-10-16T08:38:46.628731+0000","last_became_active":"2023-10-16T08:38:44.645000+0000","last_became_peered":"2023-10-16T08:38:44.645000+0000","last_unstale":"2023-10-16T08:38:46.628731+0000","last_undegraded":"2023-10-16T08:38:46.628731+0000","last_fullsized":"2023-10-16T08:38:46.628731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628628+0000","last_change":"2023-10-16T08:38:45.664252+0000","last_active":"2023-10-16T08:38:46.628628+0000","last_peered":"2023-10-16T08:38:46.628628+0000","last_clean":"2023-10-16T08:38:46.628628+0000","last_became_active":"2023-10-16T08:38:45.663538+0000","last_became_peered":"2023-10-16T08:38:45.663538+0000","last_unstale":"2023-10-16T08:38:46.628628+0000","last_undegraded":"2023-10-16T08:38:46.628628+0000","last_fullsized":"2023-10-16T08:38:46.628628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926310+0000","last_change":"2023-10-16T08:38:44.646321+0000","last_active":"2023-10-16T08:38:46.926310+0000","last_peered":"2023-10-16T08:38:46.926310+0000","last_clean":"2023-10-16T08:38:46.926310+0000","last_became_active":"2023-10-16T08:38:44.645854+0000","last_became_peered":"2023-10-16T08:38:44.645854+0000","last_unstale":"2023-10-16T08:38:46.926310+0000","last_undegraded":"2023-10-16T08:38:46.926310+0000","last_fullsized":"2023-10-16T08:38:46.926310+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:10.980982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":70988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.534}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:38:48.988 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:38:49.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.148+0000 7ff81e9b2640 1 Processor -- start 2023-10-16T08:38:49.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.148+0000 7ff81e9b2640 1 -- start start 2023-10-16T08:38:49.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.152+0000 7ff81e9b2640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff81815d180 0x7ff81806b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:49.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.152+0000 7ff81e9b2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff81815d560 0x7ff81806fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:49.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.152+0000 7ff81e9b2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff8180ad820 0x7ff8180c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:49.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.152+0000 7ff81e9b2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff818070090 con 0x7ff8180ad820 2023-10-16T08:38:49.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.152+0000 7ff81e9b2640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8180cf480 con 0x7ff81815d180 2023-10-16T08:38:49.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.152+0000 7ff81e9b2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8180cf760 con 0x7ff81815d560 2023-10-16T08:38:49.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.152+0000 7ff81cf4f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff8180ad820 0x7ff8180c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:49.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.152+0000 7ff817fff640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff81815d180 0x7ff81806b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:49.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.152+0000 7ff81cf4f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff8180ad820 0x7ff8180c9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58124/0 (socket says 172.21.15.40:58124) 2023-10-16T08:38:49.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.152+0000 7ff81cf4f640 1 -- 172.21.15.40:0/3992929076 learned_addr learned my addr 172.21.15.40:0/3992929076 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:49.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.152+0000 7ff8177fe640 1 --2- 172.21.15.40:0/3992929076 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff81815d560 0x7ff81806fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:49.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.152+0000 7ff81cf4f640 1 -- 172.21.15.40:0/3992929076 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff81815d560 msgr2=0x7ff81806fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:49.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.152+0000 7ff81cf4f640 1 --2- 172.21.15.40:0/3992929076 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff81815d560 0x7ff81806fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.152+0000 7ff81cf4f640 1 -- 172.21.15.40:0/3992929076 >> v2:172.21.15.86:3300/0 conn(0x7ff81815d180 msgr2=0x7ff81806b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:49.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.152+0000 7ff81cf4f640 1 --2- 172.21.15.40:0/3992929076 >> v2:172.21.15.86:3300/0 conn(0x7ff81815d180 0x7ff81806b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.152+0000 7ff81cf4f640 1 -- 172.21.15.40:0/3992929076 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8180c8fb0 con 0x7ff8180ad820 2023-10-16T08:38:49.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.152+0000 7ff81cf4f640 1 --2- 172.21.15.40:0/3992929076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8180ad820 0x7ff8180c9570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff81806a0d0 tx=0x7ff81001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31ffa61945e5c675 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:49.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.152+0000 7ff816ffd640 1 -- 172.21.15.40:0/3992929076 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8100aa5f0 con 0x7ff8180ad820 2023-10-16T08:38:49.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.152+0000 7ff816ffd640 1 -- 172.21.15.40:0/3992929076 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8100aa750 con 0x7ff8180ad820 2023-10-16T08:38:49.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff816ffd640 1 -- 172.21.15.40:0/3992929076 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8100aaae0 con 0x7ff8180ad820 2023-10-16T08:38:49.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff81e9b2640 1 -- 172.21.15.40:0/3992929076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8180ad820 msgr2=0x7ff8180c9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:49.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff81e9b2640 1 --2- 172.21.15.40:0/3992929076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8180ad820 0x7ff8180c9570 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff81806a0d0 tx=0x7ff81001c4b0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff81e9b2640 1 -- 172.21.15.40:0/3992929076 shutdown_connections 2023-10-16T08:38:49.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff81e9b2640 1 --2- 172.21.15.40:0/3992929076 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff8180ad820 0x7ff8180c9570 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff81e9b2640 1 --2- 172.21.15.40:0/3992929076 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff81815d560 0x7ff81806fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff81e9b2640 1 --2- 172.21.15.40:0/3992929076 >> v2:172.21.15.86:3300/0 conn(0x7ff81815d180 0x7ff81806b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff81e9b2640 1 -- 172.21.15.40:0/3992929076 >> 172.21.15.40:0/3992929076 conn(0x7ff8180ce960 msgr2=0x7ff8180ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:49.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff81e9b2640 1 -- 172.21.15.40:0/3992929076 shutdown_connections 2023-10-16T08:38:49.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff81e9b2640 1 -- 172.21.15.40:0/3992929076 wait complete. 2023-10-16T08:38:49.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff81e9b2640 1 Processor -- start 2023-10-16T08:38:49.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff81e9b2640 1 -- start start 2023-10-16T08:38:49.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff81e9b2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8180ad820 0x7ff8180c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:49.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff81e9b2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff81815d180 0x7ff8180c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:49.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff81e9b2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff81815d560 0x7ff818154a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:49.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff81e9b2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8180c8b70 con 0x7ff81815d180 2023-10-16T08:38:49.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff81e9b2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8180c5f70 con 0x7ff81815d560 2023-10-16T08:38:49.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff81e9b2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8180c61a0 con 0x7ff8180ad820 2023-10-16T08:38:49.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff8177fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff81815d180 0x7ff8180c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:49.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff8177fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff81815d180 0x7ff8180c5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58132/0 (socket says 172.21.15.40:58132) 2023-10-16T08:38:49.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff8177fe640 1 -- 172.21.15.40:0/4092314538 learned_addr learned my addr 172.21.15.40:0/4092314538 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:49.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff817fff640 1 --2- 172.21.15.40:0/4092314538 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8180ad820 0x7ff8180c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:49.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff8177fe640 1 -- 172.21.15.40:0/4092314538 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8180ad820 msgr2=0x7ff8180c6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:49.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff8177fe640 1 --2- 172.21.15.40:0/4092314538 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8180ad820 0x7ff8180c6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff8177fe640 1 -- 172.21.15.40:0/4092314538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff81815d560 msgr2=0x7ff818154a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:49.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff8177fe640 1 --2- 172.21.15.40:0/4092314538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff81815d560 0x7ff818154a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff8177fe640 1 -- 172.21.15.40:0/4092314538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8100094d0 con 0x7ff81815d180 2023-10-16T08:38:49.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff8177fe640 1 --2- 172.21.15.40:0/4092314538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff81815d180 0x7ff8180c5a30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff80000e870 tx=0x7ff80000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:49.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff814ff9640 1 -- 172.21.15.40:0/4092314538 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff800009510 con 0x7ff81815d180 2023-10-16T08:38:49.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff814ff9640 1 -- 172.21.15.40:0/4092314538 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff80000eec0 con 0x7ff81815d180 2023-10-16T08:38:49.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff81e9b2640 1 -- 172.21.15.40:0/4092314538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8180c81d0 con 0x7ff81815d180 2023-10-16T08:38:49.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff814ff9640 1 -- 172.21.15.40:0/4092314538 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff80000cb40 con 0x7ff81815d180 2023-10-16T08:38:49.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.156+0000 7ff81e9b2640 1 -- 172.21.15.40:0/4092314538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8180c8790 con 0x7ff81815d180 2023-10-16T08:38:49.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.160+0000 7ff81e9b2640 1 -- 172.21.15.40:0/4092314538 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff81804e9c0 con 0x7ff81815d180 2023-10-16T08:38:49.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.160+0000 7ff814ff9640 1 -- 172.21.15.40:0/4092314538 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff80000cca0 con 0x7ff81815d180 2023-10-16T08:38:49.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.160+0000 7ff814ff9640 1 --2- 172.21.15.40:0/4092314538 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff7f80664a0 0x7ff7f8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:49.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.160+0000 7ff814ff9640 1 -- 172.21.15.40:0/4092314538 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff800083f70 con 0x7ff81815d180 2023-10-16T08:38:49.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.160+0000 7ff817fff640 1 --2- 172.21.15.40:0/4092314538 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff7f80664a0 0x7ff7f8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:49.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.160+0000 7ff817fff640 1 --2- 172.21.15.40:0/4092314538 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff7f80664a0 0x7ff7f8068960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff808009fd0 tx=0x7ff80800b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:49.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.164+0000 7ff814ff9640 1 -- 172.21.15.40:0/4092314538 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7ff8000b53d0 con 0x7ff81815d180 2023-10-16T08:38:49.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.312+0000 7ff81e9b2640 1 -- 172.21.15.40:0/4092314538 --> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff8180c22d0 con 0x7ff7f80664a0 2023-10-16T08:38:49.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.332+0000 7ff814ff9640 1 -- 172.21.15.40:0/4092314538 <== mgr.4111 v2:172.21.15.86:6832/1188970899 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264546 (secure 0 0 0) 0x7ff8180c22d0 con 0x7ff7f80664a0 2023-10-16T08:38:49.340 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:49.341 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-16T08:38:49.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.336+0000 7ff81e9b2640 1 -- 172.21.15.40:0/4092314538 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff7f80664a0 msgr2=0x7ff7f8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:49.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.336+0000 7ff81e9b2640 1 --2- 172.21.15.40:0/4092314538 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff7f80664a0 0x7ff7f8068960 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff808009fd0 tx=0x7ff80800b040 comp rx=0 tx=0).stop 2023-10-16T08:38:49.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.336+0000 7ff81e9b2640 1 -- 172.21.15.40:0/4092314538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff81815d180 msgr2=0x7ff8180c5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:49.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.336+0000 7ff81e9b2640 1 --2- 172.21.15.40:0/4092314538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff81815d180 0x7ff8180c5a30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff80000e870 tx=0x7ff80000ec70 comp rx=0 tx=0).stop 2023-10-16T08:38:49.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.336+0000 7ff81e9b2640 1 -- 172.21.15.40:0/4092314538 shutdown_connections 2023-10-16T08:38:49.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.336+0000 7ff81e9b2640 1 --2- 172.21.15.40:0/4092314538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff81815d560 0x7ff818154a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.336+0000 7ff81e9b2640 1 --2- 172.21.15.40:0/4092314538 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff7f80664a0 0x7ff7f8068960 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.336+0000 7ff81e9b2640 1 --2- 172.21.15.40:0/4092314538 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff81815d180 0x7ff8180c5a30 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.336+0000 7ff81e9b2640 1 --2- 172.21.15.40:0/4092314538 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff8180ad820 0x7ff8180c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.336+0000 7ff81e9b2640 1 -- 172.21.15.40:0/4092314538 >> 172.21.15.40:0/4092314538 conn(0x7ff8180ce960 msgr2=0x7ff81815c410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:49.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.336+0000 7ff81e9b2640 1 -- 172.21.15.40:0/4092314538 shutdown_connections 2023-10-16T08:38:49.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.336+0000 7ff81e9b2640 1 -- 172.21.15.40:0/4092314538 wait complete. 2023-10-16T08:38:49.356 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":21,"stamp":"2023-10-16T08:38:48.938600+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":174,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219016,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755704,"statfs":{"total":773094113280,"available":772869840896,"internally_reserved":0,"allocated":2891776,"data_stored":1752740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":63,"apply_latency_ms":63,"commit_latency_ns":63000000,"apply_latency_ns":63000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.341363"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931534+0000","last_change":"2023-10-16T08:38:45.650619+0000","last_active":"2023-10-16T08:38:46.931534+0000","last_peered":"2023-10-16T08:38:46.931534+0000","last_clean":"2023-10-16T08:38:46.931534+0000","last_became_active":"2023-10-16T08:38:45.649704+0000","last_became_peered":"2023-10-16T08:38:45.649704+0000","last_unstale":"2023-10-16T08:38:46.931534+0000","last_undegraded":"2023-10-16T08:38:46.931534+0000","last_fullsized":"2023-10-16T08:38:46.931534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:58:51.599709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626698+0000","last_change":"2023-10-16T08:38:45.657659+0000","last_active":"2023-10-16T08:38:46.626698+0000","last_peered":"2023-10-16T08:38:46.626698+0000","last_clean":"2023-10-16T08:38:46.626698+0000","last_became_active":"2023-10-16T08:38:45.656884+0000","last_became_peered":"2023-10-16T08:38:45.656884+0000","last_unstale":"2023-10-16T08:38:46.626698+0000","last_undegraded":"2023-10-16T08:38:46.626698+0000","last_fullsized":"2023-10-16T08:38:46.626698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926261+0000","last_change":"2023-10-16T08:38:45.630125+0000","last_active":"2023-10-16T08:38:46.926261+0000","last_peered":"2023-10-16T08:38:46.926261+0000","last_clean":"2023-10-16T08:38:46.926261+0000","last_became_active":"2023-10-16T08:38:45.629297+0000","last_became_peered":"2023-10-16T08:38:45.629297+0000","last_unstale":"2023-10-16T08:38:46.926261+0000","last_undegraded":"2023-10-16T08:38:46.926261+0000","last_fullsized":"2023-10-16T08:38:46.926261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931028+0000","last_change":"2023-10-16T08:38:45.675996+0000","last_active":"2023-10-16T08:38:46.931028+0000","last_peered":"2023-10-16T08:38:46.931028+0000","last_clean":"2023-10-16T08:38:46.931028+0000","last_became_active":"2023-10-16T08:38:45.675327+0000","last_became_peered":"2023-10-16T08:38:45.675327+0000","last_unstale":"2023-10-16T08:38:46.931028+0000","last_undegraded":"2023-10-16T08:38:46.931028+0000","last_fullsized":"2023-10-16T08:38:46.931028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:33.190218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930419+0000","last_change":"2023-10-16T08:38:45.653550+0000","last_active":"2023-10-16T08:38:46.930419+0000","last_peered":"2023-10-16T08:38:46.930419+0000","last_clean":"2023-10-16T08:38:46.930419+0000","last_became_active":"2023-10-16T08:38:45.652808+0000","last_became_peered":"2023-10-16T08:38:45.652808+0000","last_unstale":"2023-10-16T08:38:46.930419+0000","last_undegraded":"2023-10-16T08:38:46.930419+0000","last_fullsized":"2023-10-16T08:38:46.930419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930519+0000","last_change":"2023-10-16T08:38:45.675086+0000","last_active":"2023-10-16T08:38:46.930519+0000","last_peered":"2023-10-16T08:38:46.930519+0000","last_clean":"2023-10-16T08:38:46.930519+0000","last_became_active":"2023-10-16T08:38:45.671146+0000","last_became_peered":"2023-10-16T08:38:45.671146+0000","last_unstale":"2023-10-16T08:38:46.930519+0000","last_undegraded":"2023-10-16T08:38:46.930519+0000","last_fullsized":"2023-10-16T08:38:46.930519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931145+0000","last_change":"2023-10-16T08:38:45.652595+0000","last_active":"2023-10-16T08:38:46.931145+0000","last_peered":"2023-10-16T08:38:46.931145+0000","last_clean":"2023-10-16T08:38:46.931145+0000","last_became_active":"2023-10-16T08:38:45.651848+0000","last_became_peered":"2023-10-16T08:38:45.651848+0000","last_unstale":"2023-10-16T08:38:46.931145+0000","last_undegraded":"2023-10-16T08:38:46.931145+0000","last_fullsized":"2023-10-16T08:38:46.931145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:13.453321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628116+0000","last_change":"2023-10-16T08:38:45.668858+0000","last_active":"2023-10-16T08:38:46.628116+0000","last_peered":"2023-10-16T08:38:46.628116+0000","last_clean":"2023-10-16T08:38:46.628116+0000","last_became_active":"2023-10-16T08:38:45.664702+0000","last_became_peered":"2023-10-16T08:38:45.664702+0000","last_unstale":"2023-10-16T08:38:46.628116+0000","last_undegraded":"2023-10-16T08:38:46.628116+0000","last_fullsized":"2023-10-16T08:38:46.628116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931734+0000","last_change":"2023-10-16T08:38:45.667037+0000","last_active":"2023-10-16T08:38:46.931734+0000","last_peered":"2023-10-16T08:38:46.931734+0000","last_clean":"2023-10-16T08:38:46.931734+0000","last_became_active":"2023-10-16T08:38:45.666474+0000","last_became_peered":"2023-10-16T08:38:45.666474+0000","last_unstale":"2023-10-16T08:38:46.931734+0000","last_undegraded":"2023-10-16T08:38:46.931734+0000","last_fullsized":"2023-10-16T08:38:46.931734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:11:02.634006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633786+0000","last_change":"2023-10-16T08:38:45.669859+0000","last_active":"2023-10-16T08:38:46.633786+0000","last_peered":"2023-10-16T08:38:46.633786+0000","last_clean":"2023-10-16T08:38:46.633786+0000","last_became_active":"2023-10-16T08:38:45.668278+0000","last_became_peered":"2023-10-16T08:38:45.668278+0000","last_unstale":"2023-10-16T08:38:46.633786+0000","last_undegraded":"2023-10-16T08:38:46.633786+0000","last_fullsized":"2023-10-16T08:38:46.633786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633464+0000","last_change":"2023-10-16T08:38:45.664567+0000","last_active":"2023-10-16T08:38:46.633464+0000","last_peered":"2023-10-16T08:38:46.633464+0000","last_clean":"2023-10-16T08:38:46.633464+0000","last_became_active":"2023-10-16T08:38:45.663813+0000","last_became_peered":"2023-10-16T08:38:45.663813+0000","last_unstale":"2023-10-16T08:38:46.633464+0000","last_undegraded":"2023-10-16T08:38:46.633464+0000","last_fullsized":"2023-10-16T08:38:46.633464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.932310+0000","last_change":"2023-10-16T08:38:45.675925+0000","last_active":"2023-10-16T08:38:46.932310+0000","last_peered":"2023-10-16T08:38:46.932310+0000","last_clean":"2023-10-16T08:38:46.932310+0000","last_became_active":"2023-10-16T08:38:45.675320+0000","last_became_peered":"2023-10-16T08:38:45.675320+0000","last_unstale":"2023-10-16T08:38:46.932310+0000","last_undegraded":"2023-10-16T08:38:46.932310+0000","last_fullsized":"2023-10-16T08:38:46.932310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:11.991837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628294+0000","last_change":"2023-10-16T08:38:45.668056+0000","last_active":"2023-10-16T08:38:46.628294+0000","last_peered":"2023-10-16T08:38:46.628294+0000","last_clean":"2023-10-16T08:38:46.628294+0000","last_became_active":"2023-10-16T08:38:45.667428+0000","last_became_peered":"2023-10-16T08:38:45.667428+0000","last_unstale":"2023-10-16T08:38:46.628294+0000","last_undegraded":"2023-10-16T08:38:46.628294+0000","last_fullsized":"2023-10-16T08:38:46.628294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.925984+0000","last_change":"2023-10-16T08:38:45.637518+0000","last_active":"2023-10-16T08:38:46.925984+0000","last_peered":"2023-10-16T08:38:46.925984+0000","last_clean":"2023-10-16T08:38:46.925984+0000","last_became_active":"2023-10-16T08:38:45.636785+0000","last_became_peered":"2023-10-16T08:38:45.636785+0000","last_unstale":"2023-10-16T08:38:46.925984+0000","last_undegraded":"2023-10-16T08:38:46.925984+0000","last_fullsized":"2023-10-16T08:38:46.925984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:11.870171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626028+0000","last_change":"2023-10-16T08:38:45.656433+0000","last_active":"2023-10-16T08:38:46.626028+0000","last_peered":"2023-10-16T08:38:46.626028+0000","last_clean":"2023-10-16T08:38:46.626028+0000","last_became_active":"2023-10-16T08:38:45.655395+0000","last_became_peered":"2023-10-16T08:38:45.655395+0000","last_unstale":"2023-10-16T08:38:46.626028+0000","last_undegraded":"2023-10-16T08:38:46.626028+0000","last_fullsized":"2023-10-16T08:38:46.626028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.929381+0000","last_change":"2023-10-16T08:38:45.664192+0000","last_active":"2023-10-16T08:38:46.929381+0000","last_peered":"2023-10-16T08:38:46.929381+0000","last_clean":"2023-10-16T08:38:46.929381+0000","last_became_active":"2023-10-16T08:38:45.663312+0000","last_became_peered":"2023-10-16T08:38:45.663312+0000","last_unstale":"2023-10-16T08:38:46.929381+0000","last_undegraded":"2023-10-16T08:38:46.929381+0000","last_fullsized":"2023-10-16T08:38:46.929381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930671+0000","last_change":"2023-10-16T08:38:45.679894+0000","last_active":"2023-10-16T08:38:46.930671+0000","last_peered":"2023-10-16T08:38:46.930671+0000","last_clean":"2023-10-16T08:38:46.930671+0000","last_became_active":"2023-10-16T08:38:45.676912+0000","last_became_peered":"2023-10-16T08:38:45.676912+0000","last_unstale":"2023-10-16T08:38:46.930671+0000","last_undegraded":"2023-10-16T08:38:46.930671+0000","last_fullsized":"2023-10-16T08:38:46.930671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927699+0000","last_change":"2023-10-16T08:38:45.631488+0000","last_active":"2023-10-16T08:38:46.927699+0000","last_peered":"2023-10-16T08:38:46.927699+0000","last_clean":"2023-10-16T08:38:46.927699+0000","last_became_active":"2023-10-16T08:38:45.630641+0000","last_became_peered":"2023-10-16T08:38:45.630641+0000","last_unstale":"2023-10-16T08:38:46.927699+0000","last_undegraded":"2023-10-16T08:38:46.927699+0000","last_fullsized":"2023-10-16T08:38:46.927699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:11.991837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931188+0000","last_change":"2023-10-16T08:38:45.650087+0000","last_active":"2023-10-16T08:38:46.931188+0000","last_peered":"2023-10-16T08:38:46.931188+0000","last_clean":"2023-10-16T08:38:46.931188+0000","last_became_active":"2023-10-16T08:38:45.649475+0000","last_became_peered":"2023-10-16T08:38:45.649475+0000","last_unstale":"2023-10-16T08:38:46.931188+0000","last_undegraded":"2023-10-16T08:38:46.931188+0000","last_fullsized":"2023-10-16T08:38:46.931188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:26.292137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633283+0000","last_change":"2023-10-16T08:38:45.667336+0000","last_active":"2023-10-16T08:38:46.633283+0000","last_peered":"2023-10-16T08:38:46.633283+0000","last_clean":"2023-10-16T08:38:46.633283+0000","last_became_active":"2023-10-16T08:38:45.665291+0000","last_became_peered":"2023-10-16T08:38:45.665291+0000","last_unstale":"2023-10-16T08:38:46.633283+0000","last_undegraded":"2023-10-16T08:38:46.633283+0000","last_fullsized":"2023-10-16T08:38:46.633283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926490+0000","last_change":"2023-10-16T08:38:45.654036+0000","last_active":"2023-10-16T08:38:46.926490+0000","last_peered":"2023-10-16T08:38:46.926490+0000","last_clean":"2023-10-16T08:38:46.926490+0000","last_became_active":"2023-10-16T08:38:45.652412+0000","last_became_peered":"2023-10-16T08:38:45.652412+0000","last_unstale":"2023-10-16T08:38:46.926490+0000","last_undegraded":"2023-10-16T08:38:46.926490+0000","last_fullsized":"2023-10-16T08:38:46.926490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930858+0000","last_change":"2023-10-16T08:38:45.677644+0000","last_active":"2023-10-16T08:38:46.930858+0000","last_peered":"2023-10-16T08:38:46.930858+0000","last_clean":"2023-10-16T08:38:46.930858+0000","last_became_active":"2023-10-16T08:38:45.676844+0000","last_became_peered":"2023-10-16T08:38:45.676844+0000","last_unstale":"2023-10-16T08:38:46.930858+0000","last_undegraded":"2023-10-16T08:38:46.930858+0000","last_fullsized":"2023-10-16T08:38:46.930858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628178+0000","last_change":"2023-10-16T08:38:45.663771+0000","last_active":"2023-10-16T08:38:46.628178+0000","last_peered":"2023-10-16T08:38:46.628178+0000","last_clean":"2023-10-16T08:38:46.628178+0000","last_became_active":"2023-10-16T08:38:45.662944+0000","last_became_peered":"2023-10-16T08:38:45.662944+0000","last_unstale":"2023-10-16T08:38:46.628178+0000","last_undegraded":"2023-10-16T08:38:46.628178+0000","last_fullsized":"2023-10-16T08:38:46.628178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.929737+0000","last_change":"2023-10-16T08:38:45.667398+0000","last_active":"2023-10-16T08:38:46.929737+0000","last_peered":"2023-10-16T08:38:46.929737+0000","last_clean":"2023-10-16T08:38:46.929737+0000","last_became_active":"2023-10-16T08:38:45.666641+0000","last_became_peered":"2023-10-16T08:38:45.666641+0000","last_unstale":"2023-10-16T08:38:46.929737+0000","last_undegraded":"2023-10-16T08:38:46.929737+0000","last_fullsized":"2023-10-16T08:38:46.929737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628798+0000","last_change":"2023-10-16T08:38:45.670526+0000","last_active":"2023-10-16T08:38:46.628798+0000","last_peered":"2023-10-16T08:38:46.628798+0000","last_clean":"2023-10-16T08:38:46.628798+0000","last_became_active":"2023-10-16T08:38:45.669734+0000","last_became_peered":"2023-10-16T08:38:45.669734+0000","last_unstale":"2023-10-16T08:38:46.628798+0000","last_undegraded":"2023-10-16T08:38:46.628798+0000","last_fullsized":"2023-10-16T08:38:46.628798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633809+0000","last_change":"2023-10-16T08:38:45.670429+0000","last_active":"2023-10-16T08:38:46.633809+0000","last_peered":"2023-10-16T08:38:46.633809+0000","last_clean":"2023-10-16T08:38:46.633809+0000","last_became_active":"2023-10-16T08:38:45.665972+0000","last_became_peered":"2023-10-16T08:38:45.665972+0000","last_unstale":"2023-10-16T08:38:46.633809+0000","last_undegraded":"2023-10-16T08:38:46.633809+0000","last_fullsized":"2023-10-16T08:38:46.633809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626436+0000","last_change":"2023-10-16T08:38:45.667047+0000","last_active":"2023-10-16T08:38:46.626436+0000","last_peered":"2023-10-16T08:38:46.626436+0000","last_clean":"2023-10-16T08:38:46.626436+0000","last_became_active":"2023-10-16T08:38:45.666403+0000","last_became_peered":"2023-10-16T08:38:45.666403+0000","last_unstale":"2023-10-16T08:38:46.626436+0000","last_undegraded":"2023-10-16T08:38:46.626436+0000","last_fullsized":"2023-10-16T08:38:46.626436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628642+0000","last_change":"2023-10-16T08:38:45.668007+0000","last_active":"2023-10-16T08:38:46.628642+0000","last_peered":"2023-10-16T08:38:46.628642+0000","last_clean":"2023-10-16T08:38:46.628642+0000","last_became_active":"2023-10-16T08:38:45.664572+0000","last_became_peered":"2023-10-16T08:38:45.664572+0000","last_unstale":"2023-10-16T08:38:46.628642+0000","last_undegraded":"2023-10-16T08:38:46.628642+0000","last_fullsized":"2023-10-16T08:38:46.628642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926222+0000","last_change":"2023-10-16T08:38:45.667378+0000","last_active":"2023-10-16T08:38:46.926222+0000","last_peered":"2023-10-16T08:38:46.926222+0000","last_clean":"2023-10-16T08:38:46.926222+0000","last_became_active":"2023-10-16T08:38:45.666547+0000","last_became_peered":"2023-10-16T08:38:45.666547+0000","last_unstale":"2023-10-16T08:38:46.926222+0000","last_undegraded":"2023-10-16T08:38:46.926222+0000","last_fullsized":"2023-10-16T08:38:46.926222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931201+0000","last_change":"2023-10-16T08:38:45.669071+0000","last_active":"2023-10-16T08:38:46.931201+0000","last_peered":"2023-10-16T08:38:46.931201+0000","last_clean":"2023-10-16T08:38:46.931201+0000","last_became_active":"2023-10-16T08:38:45.666591+0000","last_became_peered":"2023-10-16T08:38:45.666591+0000","last_unstale":"2023-10-16T08:38:46.931201+0000","last_undegraded":"2023-10-16T08:38:46.931201+0000","last_fullsized":"2023-10-16T08:38:46.931201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:02:39.840175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927293+0000","last_change":"2023-10-16T08:38:45.648288+0000","last_active":"2023-10-16T08:38:46.927293+0000","last_peered":"2023-10-16T08:38:46.927293+0000","last_clean":"2023-10-16T08:38:46.927293+0000","last_became_active":"2023-10-16T08:38:45.647484+0000","last_became_peered":"2023-10-16T08:38:45.647484+0000","last_unstale":"2023-10-16T08:38:46.927293+0000","last_undegraded":"2023-10-16T08:38:46.927293+0000","last_fullsized":"2023-10-16T08:38:46.927293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:20.779617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.632427+0000","last_change":"2023-10-16T08:38:45.638183+0000","last_active":"2023-10-16T08:38:46.632427+0000","last_peered":"2023-10-16T08:38:46.632427+0000","last_clean":"2023-10-16T08:38:46.632427+0000","last_became_active":"2023-10-16T08:38:45.636762+0000","last_became_peered":"2023-10-16T08:38:45.636762+0000","last_unstale":"2023-10-16T08:38:46.632427+0000","last_undegraded":"2023-10-16T08:38:46.632427+0000","last_fullsized":"2023-10-16T08:38:46.632427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.929995+0000","last_change":"2023-10-16T08:38:45.659464+0000","last_active":"2023-10-16T08:38:46.929995+0000","last_peered":"2023-10-16T08:38:46.929995+0000","last_clean":"2023-10-16T08:38:46.929995+0000","last_became_active":"2023-10-16T08:38:45.658808+0000","last_became_peered":"2023-10-16T08:38:45.658808+0000","last_unstale":"2023-10-16T08:38:46.929995+0000","last_undegraded":"2023-10-16T08:38:46.929995+0000","last_fullsized":"2023-10-16T08:38:46.929995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:25.386221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628419+0000","last_change":"2023-10-16T08:38:44.644122+0000","last_active":"2023-10-16T08:38:46.628419+0000","last_peered":"2023-10-16T08:38:46.628419+0000","last_clean":"2023-10-16T08:38:46.628419+0000","last_became_active":"2023-10-16T08:38:44.643614+0000","last_became_peered":"2023-10-16T08:38:44.643614+0000","last_unstale":"2023-10-16T08:38:46.628419+0000","last_undegraded":"2023-10-16T08:38:46.628419+0000","last_fullsized":"2023-10-16T08:38:46.628419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926570+0000","last_change":"2023-10-16T08:38:45.659854+0000","last_active":"2023-10-16T08:38:46.926570+0000","last_peered":"2023-10-16T08:38:46.926570+0000","last_clean":"2023-10-16T08:38:46.926570+0000","last_became_active":"2023-10-16T08:38:45.658409+0000","last_became_peered":"2023-10-16T08:38:45.658409+0000","last_unstale":"2023-10-16T08:38:46.926570+0000","last_undegraded":"2023-10-16T08:38:46.926570+0000","last_fullsized":"2023-10-16T08:38:46.926570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.632862+0000","last_change":"2023-10-16T08:38:44.658819+0000","last_active":"2023-10-16T08:38:46.632862+0000","last_peered":"2023-10-16T08:38:46.632862+0000","last_clean":"2023-10-16T08:38:46.632862+0000","last_became_active":"2023-10-16T08:38:44.658257+0000","last_became_peered":"2023-10-16T08:38:44.658257+0000","last_unstale":"2023-10-16T08:38:46.632862+0000","last_undegraded":"2023-10-16T08:38:46.632862+0000","last_fullsized":"2023-10-16T08:38:46.632862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:54.848445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628283+0000","last_change":"2023-10-16T08:38:44.647536+0000","last_active":"2023-10-16T08:38:46.628283+0000","last_peered":"2023-10-16T08:38:46.628283+0000","last_clean":"2023-10-16T08:38:46.628283+0000","last_became_active":"2023-10-16T08:38:44.645931+0000","last_became_peered":"2023-10-16T08:38:44.645931+0000","last_unstale":"2023-10-16T08:38:46.628283+0000","last_undegraded":"2023-10-16T08:38:46.628283+0000","last_fullsized":"2023-10-16T08:38:46.628283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628223+0000","last_change":"2023-10-16T08:38:45.662855+0000","last_active":"2023-10-16T08:38:46.628223+0000","last_peered":"2023-10-16T08:38:46.628223+0000","last_clean":"2023-10-16T08:38:46.628223+0000","last_became_active":"2023-10-16T08:38:45.662072+0000","last_became_peered":"2023-10-16T08:38:45.662072+0000","last_unstale":"2023-10-16T08:38:46.628223+0000","last_undegraded":"2023-10-16T08:38:46.628223+0000","last_fullsized":"2023-10-16T08:38:46.628223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.929953+0000","last_change":"2023-10-16T08:38:44.657651+0000","last_active":"2023-10-16T08:38:46.929953+0000","last_peered":"2023-10-16T08:38:46.929953+0000","last_clean":"2023-10-16T08:38:46.929953+0000","last_became_active":"2023-10-16T08:38:44.657065+0000","last_became_peered":"2023-10-16T08:38:44.657065+0000","last_unstale":"2023-10-16T08:38:46.929953+0000","last_undegraded":"2023-10-16T08:38:46.929953+0000","last_fullsized":"2023-10-16T08:38:46.929953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.625975+0000","last_change":"2023-10-16T08:38:45.668048+0000","last_active":"2023-10-16T08:38:46.625975+0000","last_peered":"2023-10-16T08:38:46.625975+0000","last_clean":"2023-10-16T08:38:46.625975+0000","last_became_active":"2023-10-16T08:38:45.667413+0000","last_became_peered":"2023-10-16T08:38:45.667413+0000","last_unstale":"2023-10-16T08:38:46.625975+0000","last_undegraded":"2023-10-16T08:38:46.625975+0000","last_fullsized":"2023-10-16T08:38:46.625975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633561+0000","last_change":"2023-10-16T08:38:44.663188+0000","last_active":"2023-10-16T08:38:46.633561+0000","last_peered":"2023-10-16T08:38:46.633561+0000","last_clean":"2023-10-16T08:38:46.633561+0000","last_became_active":"2023-10-16T08:38:44.662383+0000","last_became_peered":"2023-10-16T08:38:44.662383+0000","last_unstale":"2023-10-16T08:38:46.633561+0000","last_undegraded":"2023-10-16T08:38:46.633561+0000","last_fullsized":"2023-10-16T08:38:46.633561+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:24.375366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930913+0000","last_change":"2023-10-16T08:38:45.679271+0000","last_active":"2023-10-16T08:38:46.930913+0000","last_peered":"2023-10-16T08:38:46.930913+0000","last_clean":"2023-10-16T08:38:46.930913+0000","last_became_active":"2023-10-16T08:38:45.678050+0000","last_became_peered":"2023-10-16T08:38:45.678050+0000","last_unstale":"2023-10-16T08:38:46.930913+0000","last_undegraded":"2023-10-16T08:38:46.930913+0000","last_fullsized":"2023-10-16T08:38:46.930913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931150+0000","last_change":"2023-10-16T08:38:44.653478+0000","last_active":"2023-10-16T08:38:46.931150+0000","last_peered":"2023-10-16T08:38:46.931150+0000","last_clean":"2023-10-16T08:38:46.931150+0000","last_became_active":"2023-10-16T08:38:44.652618+0000","last_became_peered":"2023-10-16T08:38:44.652618+0000","last_unstale":"2023-10-16T08:38:46.931150+0000","last_undegraded":"2023-10-16T08:38:46.931150+0000","last_fullsized":"2023-10-16T08:38:46.931150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926242+0000","last_change":"2023-10-16T08:38:45.656603+0000","last_active":"2023-10-16T08:38:46.926242+0000","last_peered":"2023-10-16T08:38:46.926242+0000","last_clean":"2023-10-16T08:38:46.926242+0000","last_became_active":"2023-10-16T08:38:45.655213+0000","last_became_peered":"2023-10-16T08:38:45.655213+0000","last_unstale":"2023-10-16T08:38:46.926242+0000","last_undegraded":"2023-10-16T08:38:46.926242+0000","last_fullsized":"2023-10-16T08:38:46.926242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626622+0000","last_change":"2023-10-16T08:38:44.651637+0000","last_active":"2023-10-16T08:38:46.626622+0000","last_peered":"2023-10-16T08:38:46.626622+0000","last_clean":"2023-10-16T08:38:46.626622+0000","last_became_active":"2023-10-16T08:38:44.651359+0000","last_became_peered":"2023-10-16T08:38:44.651359+0000","last_unstale":"2023-10-16T08:38:46.626622+0000","last_undegraded":"2023-10-16T08:38:46.626622+0000","last_fullsized":"2023-10-16T08:38:46.626622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:10.859316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.932633+0000","last_change":"2023-10-16T08:38:45.661741+0000","last_active":"2023-10-16T08:38:46.932633+0000","last_peered":"2023-10-16T08:38:46.932633+0000","last_clean":"2023-10-16T08:38:46.932633+0000","last_became_active":"2023-10-16T08:38:45.660611+0000","last_became_peered":"2023-10-16T08:38:45.660611+0000","last_unstale":"2023-10-16T08:38:46.932633+0000","last_undegraded":"2023-10-16T08:38:46.932633+0000","last_fullsized":"2023-10-16T08:38:46.932633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:55.859300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633359+0000","last_change":"2023-10-16T08:38:44.647593+0000","last_active":"2023-10-16T08:38:46.633359+0000","last_peered":"2023-10-16T08:38:46.633359+0000","last_clean":"2023-10-16T08:38:46.633359+0000","last_became_active":"2023-10-16T08:38:44.647001+0000","last_became_peered":"2023-10-16T08:38:44.647001+0000","last_unstale":"2023-10-16T08:38:46.633359+0000","last_undegraded":"2023-10-16T08:38:46.633359+0000","last_fullsized":"2023-10-16T08:38:46.633359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930291+0000","last_change":"2023-10-16T08:38:45.668629+0000","last_active":"2023-10-16T08:38:46.930291+0000","last_peered":"2023-10-16T08:38:46.930291+0000","last_clean":"2023-10-16T08:38:46.930291+0000","last_became_active":"2023-10-16T08:38:45.667334+0000","last_became_peered":"2023-10-16T08:38:45.667334+0000","last_unstale":"2023-10-16T08:38:46.930291+0000","last_undegraded":"2023-10-16T08:38:46.930291+0000","last_fullsized":"2023-10-16T08:38:46.930291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926736+0000","last_change":"2023-10-16T08:38:44.655832+0000","last_active":"2023-10-16T08:38:46.926736+0000","last_peered":"2023-10-16T08:38:46.926736+0000","last_clean":"2023-10-16T08:38:46.926736+0000","last_became_active":"2023-10-16T08:38:44.655469+0000","last_became_peered":"2023-10-16T08:38:44.655469+0000","last_unstale":"2023-10-16T08:38:46.926736+0000","last_undegraded":"2023-10-16T08:38:46.926736+0000","last_fullsized":"2023-10-16T08:38:46.926736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:19.768762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926112+0000","last_change":"2023-10-16T08:38:45.653688+0000","last_active":"2023-10-16T08:38:46.926112+0000","last_peered":"2023-10-16T08:38:46.926112+0000","last_clean":"2023-10-16T08:38:46.926112+0000","last_became_active":"2023-10-16T08:38:45.650563+0000","last_became_peered":"2023-10-16T08:38:45.650563+0000","last_unstale":"2023-10-16T08:38:46.926112+0000","last_undegraded":"2023-10-16T08:38:46.926112+0000","last_fullsized":"2023-10-16T08:38:46.926112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:29.539285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931330+0000","last_change":"2023-10-16T08:38:44.662816+0000","last_active":"2023-10-16T08:38:46.931330+0000","last_peered":"2023-10-16T08:38:46.931330+0000","last_clean":"2023-10-16T08:38:46.931330+0000","last_became_active":"2023-10-16T08:38:44.662110+0000","last_became_peered":"2023-10-16T08:38:44.662110+0000","last_unstale":"2023-10-16T08:38:46.931330+0000","last_undegraded":"2023-10-16T08:38:46.931330+0000","last_fullsized":"2023-10-16T08:38:46.931330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:00.201724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930788+0000","last_change":"2023-10-16T08:38:45.662166+0000","last_active":"2023-10-16T08:38:46.930788+0000","last_peered":"2023-10-16T08:38:46.930788+0000","last_clean":"2023-10-16T08:38:46.930788+0000","last_became_active":"2023-10-16T08:38:45.661595+0000","last_became_peered":"2023-10-16T08:38:45.661595+0000","last_unstale":"2023-10-16T08:38:46.930788+0000","last_undegraded":"2023-10-16T08:38:46.930788+0000","last_fullsized":"2023-10-16T08:38:46.930788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:02:39.840175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.632731+0000","last_change":"2023-10-16T08:38:44.657766+0000","last_active":"2023-10-16T08:38:46.632731+0000","last_peered":"2023-10-16T08:38:46.632731+0000","last_clean":"2023-10-16T08:38:46.632731+0000","last_became_active":"2023-10-16T08:38:44.657153+0000","last_became_peered":"2023-10-16T08:38:44.657153+0000","last_unstale":"2023-10-16T08:38:46.632731+0000","last_undegraded":"2023-10-16T08:38:46.632731+0000","last_fullsized":"2023-10-16T08:38:46.632731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.632435+0000","last_change":"2023-10-16T08:38:45.637591+0000","last_active":"2023-10-16T08:38:46.632435+0000","last_peered":"2023-10-16T08:38:46.632435+0000","last_clean":"2023-10-16T08:38:46.632435+0000","last_became_active":"2023-10-16T08:38:45.635633+0000","last_became_peered":"2023-10-16T08:38:45.635633+0000","last_unstale":"2023-10-16T08:38:46.632435+0000","last_undegraded":"2023-10-16T08:38:46.632435+0000","last_fullsized":"2023-10-16T08:38:46.632435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927363+0000","last_change":"2023-10-16T08:38:44.638297+0000","last_active":"2023-10-16T08:38:46.927363+0000","last_peered":"2023-10-16T08:38:46.927363+0000","last_clean":"2023-10-16T08:38:46.927363+0000","last_became_active":"2023-10-16T08:38:44.637640+0000","last_became_peered":"2023-10-16T08:38:44.637640+0000","last_unstale":"2023-10-16T08:38:46.927363+0000","last_undegraded":"2023-10-16T08:38:46.927363+0000","last_fullsized":"2023-10-16T08:38:46.927363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:16.003943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931498+0000","last_change":"2023-10-16T08:38:45.657286+0000","last_active":"2023-10-16T08:38:46.931498+0000","last_peered":"2023-10-16T08:38:46.931498+0000","last_clean":"2023-10-16T08:38:46.931498+0000","last_became_active":"2023-10-16T08:38:45.656651+0000","last_became_peered":"2023-10-16T08:38:45.656651+0000","last_unstale":"2023-10-16T08:38:46.931498+0000","last_undegraded":"2023-10-16T08:38:46.931498+0000","last_fullsized":"2023-10-16T08:38:46.931498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:28.052483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.932737+0000","last_change":"2023-10-16T08:38:44.648434+0000","last_active":"2023-10-16T08:38:46.932737+0000","last_peered":"2023-10-16T08:38:46.932737+0000","last_clean":"2023-10-16T08:38:46.932737+0000","last_became_active":"2023-10-16T08:38:44.647887+0000","last_became_peered":"2023-10-16T08:38:44.647887+0000","last_unstale":"2023-10-16T08:38:46.932737+0000","last_undegraded":"2023-10-16T08:38:46.932737+0000","last_fullsized":"2023-10-16T08:38:46.932737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930414+0000","last_change":"2023-10-16T08:38:45.661965+0000","last_active":"2023-10-16T08:38:46.930414+0000","last_peered":"2023-10-16T08:38:46.930414+0000","last_clean":"2023-10-16T08:38:46.930414+0000","last_became_active":"2023-10-16T08:38:45.656750+0000","last_became_peered":"2023-10-16T08:38:45.656750+0000","last_unstale":"2023-10-16T08:38:46.930414+0000","last_undegraded":"2023-10-16T08:38:46.930414+0000","last_fullsized":"2023-10-16T08:38:46.930414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927746+0000","last_change":"2023-10-16T08:38:44.647524+0000","last_active":"2023-10-16T08:38:46.927746+0000","last_peered":"2023-10-16T08:38:46.927746+0000","last_clean":"2023-10-16T08:38:46.927746+0000","last_became_active":"2023-10-16T08:38:44.646042+0000","last_became_peered":"2023-10-16T08:38:44.646042+0000","last_unstale":"2023-10-16T08:38:46.927746+0000","last_undegraded":"2023-10-16T08:38:46.927746+0000","last_fullsized":"2023-10-16T08:38:46.927746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:24.375366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931909+0000","last_change":"2023-10-16T08:38:45.670969+0000","last_active":"2023-10-16T08:38:46.931909+0000","last_peered":"2023-10-16T08:38:46.931909+0000","last_clean":"2023-10-16T08:38:46.931909+0000","last_became_active":"2023-10-16T08:38:45.670415+0000","last_became_peered":"2023-10-16T08:38:45.670415+0000","last_unstale":"2023-10-16T08:38:46.931909+0000","last_undegraded":"2023-10-16T08:38:46.931909+0000","last_fullsized":"2023-10-16T08:38:46.931909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:15:00.472153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626608+0000","last_change":"2023-10-16T08:38:44.655744+0000","last_active":"2023-10-16T08:38:46.626608+0000","last_peered":"2023-10-16T08:38:46.626608+0000","last_clean":"2023-10-16T08:38:46.626608+0000","last_became_active":"2023-10-16T08:38:44.655260+0000","last_became_peered":"2023-10-16T08:38:44.655260+0000","last_unstale":"2023-10-16T08:38:46.626608+0000","last_undegraded":"2023-10-16T08:38:46.626608+0000","last_fullsized":"2023-10-16T08:38:46.626608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:54.848445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931710+0000","last_change":"2023-10-16T08:38:45.668230+0000","last_active":"2023-10-16T08:38:46.931710+0000","last_peered":"2023-10-16T08:38:46.931710+0000","last_clean":"2023-10-16T08:38:46.931710+0000","last_became_active":"2023-10-16T08:38:45.667627+0000","last_became_peered":"2023-10-16T08:38:45.667627+0000","last_unstale":"2023-10-16T08:38:46.931710+0000","last_undegraded":"2023-10-16T08:38:46.931710+0000","last_fullsized":"2023-10-16T08:38:46.931710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:28.052483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927508+0000","last_change":"2023-10-16T08:38:44.662118+0000","last_active":"2023-10-16T08:38:46.927508+0000","last_peered":"2023-10-16T08:38:46.927508+0000","last_clean":"2023-10-16T08:38:46.927508+0000","last_became_active":"2023-10-16T08:38:44.661736+0000","last_became_peered":"2023-10-16T08:38:44.661736+0000","last_unstale":"2023-10-16T08:38:46.927508+0000","last_undegraded":"2023-10-16T08:38:46.927508+0000","last_fullsized":"2023-10-16T08:38:46.927508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:49:40.967349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930619+0000","last_change":"2023-10-16T08:38:45.667423+0000","last_active":"2023-10-16T08:38:46.930619+0000","last_peered":"2023-10-16T08:38:46.930619+0000","last_clean":"2023-10-16T08:38:46.930619+0000","last_became_active":"2023-10-16T08:38:45.666390+0000","last_became_peered":"2023-10-16T08:38:45.666390+0000","last_unstale":"2023-10-16T08:38:46.930619+0000","last_undegraded":"2023-10-16T08:38:46.930619+0000","last_fullsized":"2023-10-16T08:38:46.930619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:33.190218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-16T08:38:45.628438+0000","last_change":"2023-10-16T08:38:34.313511+0000","last_active":"2023-10-16T08:38:45.628438+0000","last_peered":"2023-10-16T08:38:45.628438+0000","last_clean":"2023-10-16T08:38:45.628438+0000","last_became_active":"2023-10-16T08:38:34.310944+0000","last_became_peered":"2023-10-16T08:38:34.310944+0000","last_unstale":"2023-10-16T08:38:45.628438+0000","last_undegraded":"2023-10-16T08:38:45.628438+0000","last_fullsized":"2023-10-16T08:38:45.628438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:33.264692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:33.264692+0000","last_clean_scrub_stamp":"2023-10-16T08:38:33.264692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:50.170461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930271+0000","last_change":"2023-10-16T08:38:44.655223+0000","last_active":"2023-10-16T08:38:46.930271+0000","last_peered":"2023-10-16T08:38:46.930271+0000","last_clean":"2023-10-16T08:38:46.930271+0000","last_became_active":"2023-10-16T08:38:44.654277+0000","last_became_peered":"2023-10-16T08:38:44.654277+0000","last_unstale":"2023-10-16T08:38:46.930271+0000","last_undegraded":"2023-10-16T08:38:46.930271+0000","last_fullsized":"2023-10-16T08:38:46.930271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:03.462433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628392+0000","last_change":"2023-10-16T08:38:45.679868+0000","last_active":"2023-10-16T08:38:46.628392+0000","last_peered":"2023-10-16T08:38:46.628392+0000","last_clean":"2023-10-16T08:38:46.628392+0000","last_became_active":"2023-10-16T08:38:45.679409+0000","last_became_peered":"2023-10-16T08:38:45.679409+0000","last_unstale":"2023-10-16T08:38:46.628392+0000","last_undegraded":"2023-10-16T08:38:46.628392+0000","last_fullsized":"2023-10-16T08:38:46.628392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930490+0000","last_change":"2023-10-16T08:38:44.652152+0000","last_active":"2023-10-16T08:38:46.930490+0000","last_peered":"2023-10-16T08:38:46.930490+0000","last_clean":"2023-10-16T08:38:46.930490+0000","last_became_active":"2023-10-16T08:38:44.651534+0000","last_became_peered":"2023-10-16T08:38:44.651534+0000","last_unstale":"2023-10-16T08:38:46.930490+0000","last_undegraded":"2023-10-16T08:38:46.930490+0000","last_fullsized":"2023-10-16T08:38:46.930490+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926115+0000","last_change":"2023-10-16T08:38:45.651786+0000","last_active":"2023-10-16T08:38:46.926115+0000","last_peered":"2023-10-16T08:38:46.926115+0000","last_clean":"2023-10-16T08:38:46.926115+0000","last_became_active":"2023-10-16T08:38:45.650715+0000","last_became_peered":"2023-10-16T08:38:45.650715+0000","last_unstale":"2023-10-16T08:38:46.926115+0000","last_undegraded":"2023-10-16T08:38:46.926115+0000","last_fullsized":"2023-10-16T08:38:46.926115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:29.799799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927326+0000","last_change":"2023-10-16T08:38:44.645933+0000","last_active":"2023-10-16T08:38:46.927326+0000","last_peered":"2023-10-16T08:38:46.927326+0000","last_clean":"2023-10-16T08:38:46.927326+0000","last_became_active":"2023-10-16T08:38:44.645451+0000","last_became_peered":"2023-10-16T08:38:44.645451+0000","last_unstale":"2023-10-16T08:38:46.927326+0000","last_undegraded":"2023-10-16T08:38:46.927326+0000","last_fullsized":"2023-10-16T08:38:46.927326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.625977+0000","last_change":"2023-10-16T08:38:45.641499+0000","last_active":"2023-10-16T08:38:46.625977+0000","last_peered":"2023-10-16T08:38:46.625977+0000","last_clean":"2023-10-16T08:38:46.625977+0000","last_became_active":"2023-10-16T08:38:45.640765+0000","last_became_peered":"2023-10-16T08:38:45.640765+0000","last_unstale":"2023-10-16T08:38:46.625977+0000","last_undegraded":"2023-10-16T08:38:46.625977+0000","last_fullsized":"2023-10-16T08:38:46.625977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931417+0000","last_change":"2023-10-16T08:38:44.655401+0000","last_active":"2023-10-16T08:38:46.931417+0000","last_peered":"2023-10-16T08:38:46.931417+0000","last_clean":"2023-10-16T08:38:46.931417+0000","last_became_active":"2023-10-16T08:38:44.654622+0000","last_became_peered":"2023-10-16T08:38:44.654622+0000","last_unstale":"2023-10-16T08:38:46.931417+0000","last_undegraded":"2023-10-16T08:38:46.931417+0000","last_fullsized":"2023-10-16T08:38:46.931417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:48.705915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.932454+0000","last_change":"2023-10-16T08:38:45.659377+0000","last_active":"2023-10-16T08:38:46.932454+0000","last_peered":"2023-10-16T08:38:46.932454+0000","last_clean":"2023-10-16T08:38:46.932454+0000","last_became_active":"2023-10-16T08:38:45.658594+0000","last_became_peered":"2023-10-16T08:38:45.658594+0000","last_unstale":"2023-10-16T08:38:46.932454+0000","last_undegraded":"2023-10-16T08:38:46.932454+0000","last_fullsized":"2023-10-16T08:38:46.932454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:56.738058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626155+0000","last_change":"2023-10-16T08:38:44.634158+0000","last_active":"2023-10-16T08:38:46.626155+0000","last_peered":"2023-10-16T08:38:46.626155+0000","last_clean":"2023-10-16T08:38:46.626155+0000","last_became_active":"2023-10-16T08:38:44.633854+0000","last_became_peered":"2023-10-16T08:38:44.633854+0000","last_unstale":"2023-10-16T08:38:46.626155+0000","last_undegraded":"2023-10-16T08:38:46.626155+0000","last_fullsized":"2023-10-16T08:38:46.626155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:55.727203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931328+0000","last_change":"2023-10-16T08:38:45.675931+0000","last_active":"2023-10-16T08:38:46.931328+0000","last_peered":"2023-10-16T08:38:46.931328+0000","last_clean":"2023-10-16T08:38:46.931328+0000","last_became_active":"2023-10-16T08:38:45.674962+0000","last_became_peered":"2023-10-16T08:38:45.674962+0000","last_unstale":"2023-10-16T08:38:46.931328+0000","last_undegraded":"2023-10-16T08:38:46.931328+0000","last_fullsized":"2023-10-16T08:38:46.931328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:13.453321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931397+0000","last_change":"2023-10-16T08:38:44.655693+0000","last_active":"2023-10-16T08:38:46.931397+0000","last_peered":"2023-10-16T08:38:46.931397+0000","last_clean":"2023-10-16T08:38:46.931397+0000","last_became_active":"2023-10-16T08:38:44.655001+0000","last_became_peered":"2023-10-16T08:38:44.655001+0000","last_unstale":"2023-10-16T08:38:46.931397+0000","last_undegraded":"2023-10-16T08:38:46.931397+0000","last_fullsized":"2023-10-16T08:38:46.931397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:30.649639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626084+0000","last_change":"2023-10-16T08:38:45.639658+0000","last_active":"2023-10-16T08:38:46.626084+0000","last_peered":"2023-10-16T08:38:46.626084+0000","last_clean":"2023-10-16T08:38:46.626084+0000","last_became_active":"2023-10-16T08:38:45.638702+0000","last_became_peered":"2023-10-16T08:38:45.638702+0000","last_unstale":"2023-10-16T08:38:46.626084+0000","last_undegraded":"2023-10-16T08:38:46.626084+0000","last_fullsized":"2023-10-16T08:38:46.626084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626687+0000","last_change":"2023-10-16T08:38:44.648715+0000","last_active":"2023-10-16T08:38:46.626687+0000","last_peered":"2023-10-16T08:38:46.626687+0000","last_clean":"2023-10-16T08:38:46.626687+0000","last_became_active":"2023-10-16T08:38:44.648312+0000","last_became_peered":"2023-10-16T08:38:44.648312+0000","last_unstale":"2023-10-16T08:38:46.626687+0000","last_undegraded":"2023-10-16T08:38:46.626687+0000","last_fullsized":"2023-10-16T08:38:46.626687+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628807+0000","last_change":"2023-10-16T08:38:45.663402+0000","last_active":"2023-10-16T08:38:46.628807+0000","last_peered":"2023-10-16T08:38:46.628807+0000","last_clean":"2023-10-16T08:38:46.628807+0000","last_became_active":"2023-10-16T08:38:45.662442+0000","last_became_peered":"2023-10-16T08:38:45.662442+0000","last_unstale":"2023-10-16T08:38:46.628807+0000","last_undegraded":"2023-10-16T08:38:46.628807+0000","last_fullsized":"2023-10-16T08:38:46.628807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633639+0000","last_change":"2023-10-16T08:38:44.650015+0000","last_active":"2023-10-16T08:38:46.633639+0000","last_peered":"2023-10-16T08:38:46.633639+0000","last_clean":"2023-10-16T08:38:46.633639+0000","last_became_active":"2023-10-16T08:38:44.648155+0000","last_became_peered":"2023-10-16T08:38:44.648155+0000","last_unstale":"2023-10-16T08:38:46.633639+0000","last_undegraded":"2023-10-16T08:38:46.633639+0000","last_fullsized":"2023-10-16T08:38:46.633639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926726+0000","last_change":"2023-10-16T08:38:45.637568+0000","last_active":"2023-10-16T08:38:46.926726+0000","last_peered":"2023-10-16T08:38:46.926726+0000","last_clean":"2023-10-16T08:38:46.926726+0000","last_became_active":"2023-10-16T08:38:45.636751+0000","last_became_peered":"2023-10-16T08:38:45.636751+0000","last_unstale":"2023-10-16T08:38:46.926726+0000","last_undegraded":"2023-10-16T08:38:46.926726+0000","last_fullsized":"2023-10-16T08:38:46.926726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930247+0000","last_change":"2023-10-16T08:38:44.649257+0000","last_active":"2023-10-16T08:38:46.930247+0000","last_peered":"2023-10-16T08:38:46.930247+0000","last_clean":"2023-10-16T08:38:46.930247+0000","last_became_active":"2023-10-16T08:38:44.646990+0000","last_became_peered":"2023-10-16T08:38:44.646990+0000","last_unstale":"2023-10-16T08:38:46.930247+0000","last_undegraded":"2023-10-16T08:38:46.930247+0000","last_fullsized":"2023-10-16T08:38:46.930247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930389+0000","last_change":"2023-10-16T08:38:45.648228+0000","last_active":"2023-10-16T08:38:46.930389+0000","last_peered":"2023-10-16T08:38:46.930389+0000","last_clean":"2023-10-16T08:38:46.930389+0000","last_became_active":"2023-10-16T08:38:45.647449+0000","last_became_peered":"2023-10-16T08:38:45.647449+0000","last_unstale":"2023-10-16T08:38:46.930389+0000","last_undegraded":"2023-10-16T08:38:46.930389+0000","last_fullsized":"2023-10-16T08:38:46.930389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:20.779617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626045+0000","last_change":"2023-10-16T08:38:44.631354+0000","last_active":"2023-10-16T08:38:46.626045+0000","last_peered":"2023-10-16T08:38:46.626045+0000","last_clean":"2023-10-16T08:38:46.626045+0000","last_became_active":"2023-10-16T08:38:44.630451+0000","last_became_peered":"2023-10-16T08:38:44.630451+0000","last_unstale":"2023-10-16T08:38:46.626045+0000","last_undegraded":"2023-10-16T08:38:46.626045+0000","last_fullsized":"2023-10-16T08:38:46.626045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:10.980982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628323+0000","last_change":"2023-10-16T08:38:45.663898+0000","last_active":"2023-10-16T08:38:46.628323+0000","last_peered":"2023-10-16T08:38:46.628323+0000","last_clean":"2023-10-16T08:38:46.628323+0000","last_became_active":"2023-10-16T08:38:45.663100+0000","last_became_peered":"2023-10-16T08:38:45.663100+0000","last_unstale":"2023-10-16T08:38:46.628323+0000","last_undegraded":"2023-10-16T08:38:46.628323+0000","last_fullsized":"2023-10-16T08:38:46.628323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626257+0000","last_change":"2023-10-16T08:38:44.646987+0000","last_active":"2023-10-16T08:38:46.626257+0000","last_peered":"2023-10-16T08:38:46.626257+0000","last_clean":"2023-10-16T08:38:46.626257+0000","last_became_active":"2023-10-16T08:38:44.646580+0000","last_became_peered":"2023-10-16T08:38:44.646580+0000","last_unstale":"2023-10-16T08:38:46.626257+0000","last_undegraded":"2023-10-16T08:38:46.626257+0000","last_fullsized":"2023-10-16T08:38:46.626257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628448+0000","last_change":"2023-10-16T08:38:45.662894+0000","last_active":"2023-10-16T08:38:46.628448+0000","last_peered":"2023-10-16T08:38:46.628448+0000","last_clean":"2023-10-16T08:38:46.628448+0000","last_became_active":"2023-10-16T08:38:45.662226+0000","last_became_peered":"2023-10-16T08:38:45.662226+0000","last_unstale":"2023-10-16T08:38:46.628448+0000","last_undegraded":"2023-10-16T08:38:46.628448+0000","last_fullsized":"2023-10-16T08:38:46.628448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931043+0000","last_change":"2023-10-16T08:38:44.660571+0000","last_active":"2023-10-16T08:38:46.931043+0000","last_peered":"2023-10-16T08:38:46.931043+0000","last_clean":"2023-10-16T08:38:46.931043+0000","last_became_active":"2023-10-16T08:38:44.659976+0000","last_became_peered":"2023-10-16T08:38:44.659976+0000","last_unstale":"2023-10-16T08:38:46.931043+0000","last_undegraded":"2023-10-16T08:38:46.931043+0000","last_fullsized":"2023-10-16T08:38:46.931043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.632966+0000","last_change":"2023-10-16T08:38:45.665200+0000","last_active":"2023-10-16T08:38:46.632966+0000","last_peered":"2023-10-16T08:38:46.632966+0000","last_clean":"2023-10-16T08:38:46.632966+0000","last_became_active":"2023-10-16T08:38:45.664616+0000","last_became_peered":"2023-10-16T08:38:45.664616+0000","last_unstale":"2023-10-16T08:38:46.632966+0000","last_undegraded":"2023-10-16T08:38:46.632966+0000","last_fullsized":"2023-10-16T08:38:46.632966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.933149+0000","last_change":"2023-10-16T08:38:45.680892+0000","last_active":"2023-10-16T08:38:46.933149+0000","last_peered":"2023-10-16T08:38:46.933149+0000","last_clean":"2023-10-16T08:38:46.933149+0000","last_became_active":"2023-10-16T08:38:45.680231+0000","last_became_peered":"2023-10-16T08:38:45.680231+0000","last_unstale":"2023-10-16T08:38:46.933149+0000","last_undegraded":"2023-10-16T08:38:46.933149+0000","last_fullsized":"2023-10-16T08:38:46.933149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:58:51.599709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.929904+0000","last_change":"2023-10-16T08:38:44.664269+0000","last_active":"2023-10-16T08:38:46.929904+0000","last_peered":"2023-10-16T08:38:46.929904+0000","last_clean":"2023-10-16T08:38:46.929904+0000","last_became_active":"2023-10-16T08:38:44.663674+0000","last_became_peered":"2023-10-16T08:38:44.663674+0000","last_unstale":"2023-10-16T08:38:46.929904+0000","last_undegraded":"2023-10-16T08:38:46.929904+0000","last_fullsized":"2023-10-16T08:38:46.929904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931372+0000","last_change":"2023-10-16T08:38:45.680890+0000","last_active":"2023-10-16T08:38:46.931372+0000","last_peered":"2023-10-16T08:38:46.931372+0000","last_clean":"2023-10-16T08:38:46.931372+0000","last_became_active":"2023-10-16T08:38:45.680266+0000","last_became_peered":"2023-10-16T08:38:45.680266+0000","last_unstale":"2023-10-16T08:38:46.931372+0000","last_undegraded":"2023-10-16T08:38:46.931372+0000","last_fullsized":"2023-10-16T08:38:46.931372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:26.292137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930850+0000","last_change":"2023-10-16T08:38:44.653447+0000","last_active":"2023-10-16T08:38:46.930850+0000","last_peered":"2023-10-16T08:38:46.930850+0000","last_clean":"2023-10-16T08:38:46.930850+0000","last_became_active":"2023-10-16T08:38:44.652658+0000","last_became_peered":"2023-10-16T08:38:44.652658+0000","last_unstale":"2023-10-16T08:38:46.930850+0000","last_undegraded":"2023-10-16T08:38:46.930850+0000","last_fullsized":"2023-10-16T08:38:46.930850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:22.350066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926151+0000","last_change":"2023-10-16T08:38:45.665041+0000","last_active":"2023-10-16T08:38:46.926151+0000","last_peered":"2023-10-16T08:38:46.926151+0000","last_clean":"2023-10-16T08:38:46.926151+0000","last_became_active":"2023-10-16T08:38:45.664274+0000","last_became_peered":"2023-10-16T08:38:45.664274+0000","last_unstale":"2023-10-16T08:38:46.926151+0000","last_undegraded":"2023-10-16T08:38:46.926151+0000","last_fullsized":"2023-10-16T08:38:46.926151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628731+0000","last_change":"2023-10-16T08:38:44.645754+0000","last_active":"2023-10-16T08:38:46.628731+0000","last_peered":"2023-10-16T08:38:46.628731+0000","last_clean":"2023-10-16T08:38:46.628731+0000","last_became_active":"2023-10-16T08:38:44.645000+0000","last_became_peered":"2023-10-16T08:38:44.645000+0000","last_unstale":"2023-10-16T08:38:46.628731+0000","last_undegraded":"2023-10-16T08:38:46.628731+0000","last_fullsized":"2023-10-16T08:38:46.628731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628628+0000","last_change":"2023-10-16T08:38:45.664252+0000","last_active":"2023-10-16T08:38:46.628628+0000","last_peered":"2023-10-16T08:38:46.628628+0000","last_clean":"2023-10-16T08:38:46.628628+0000","last_became_active":"2023-10-16T08:38:45.663538+0000","last_became_peered":"2023-10-16T08:38:45.663538+0000","last_unstale":"2023-10-16T08:38:46.628628+0000","last_undegraded":"2023-10-16T08:38:46.628628+0000","last_fullsized":"2023-10-16T08:38:46.628628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926310+0000","last_change":"2023-10-16T08:38:44.646321+0000","last_active":"2023-10-16T08:38:46.926310+0000","last_peered":"2023-10-16T08:38:46.926310+0000","last_clean":"2023-10-16T08:38:46.926310+0000","last_became_active":"2023-10-16T08:38:44.645854+0000","last_became_peered":"2023-10-16T08:38:44.645854+0000","last_unstale":"2023-10-16T08:38:46.926310+0000","last_undegraded":"2023-10-16T08:38:46.926310+0000","last_fullsized":"2023-10-16T08:38:46.926310+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:10.980982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574853,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80600000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57699999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0569999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95799999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.095}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53300000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58599999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92200000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.131}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0960000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":70988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80800000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1559999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.181}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2330000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":13,"apply_latency_ms":13,"commit_latency_ns":13000000,"apply_latency_ns":13000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.534}]}]},{"osd":2,"up_from":13,"seq":55834574855,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.82499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1639999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0149999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1339999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63100000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98099999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.089}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.242}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92300000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.294}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:38:49.361 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-16T08:38:49.362 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-16T08:38:49.362 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-16T08:38:49.362 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-16T08:38:49.363 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-16T08:38:49.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.504+0000 7f7f02fbc640 1 Processor -- start 2023-10-16T08:38:49.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.504+0000 7f7f02fbc640 1 -- start start 2023-10-16T08:38:49.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.504+0000 7f7f02fbc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7efc15af40 0x7f7efc0cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:49.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.504+0000 7f7f02fbc640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7efc0cb5f0 0x7f7efc0714f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:49.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.504+0000 7f7f02fbc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7efc071a30 0x7f7efc15e890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:49.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.504+0000 7f7f02fbc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7efc06df30 con 0x7f7efc071a30 2023-10-16T08:38:49.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.504+0000 7f7f02fbc640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7efc06d2b0 con 0x7f7efc0cb5f0 2023-10-16T08:38:49.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.504+0000 7f7f02fbc640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc06d590 con 0x7f7efc15af40 2023-10-16T08:38:49.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.504+0000 7f7f01559640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7efc071a30 0x7f7efc15e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:49.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.504+0000 7f7f01559640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7efc071a30 0x7f7efc15e890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58140/0 (socket says 172.21.15.40:58140) 2023-10-16T08:38:49.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.504+0000 7f7f01559640 1 -- 172.21.15.40:0/1360975235 learned_addr learned my addr 172.21.15.40:0/1360975235 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:49.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.504+0000 7f7efbfff640 1 --2- 172.21.15.40:0/1360975235 >> v2:172.21.15.86:3300/0 conn(0x7f7efc0cb5f0 0x7f7efc0714f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:49.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.504+0000 7f7f00d58640 1 --2- 172.21.15.40:0/1360975235 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7efc15af40 0x7f7efc0cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:49.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.504+0000 7f7f01559640 1 -- 172.21.15.40:0/1360975235 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7efc15af40 msgr2=0x7f7efc0cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:49.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.504+0000 7f7f01559640 1 --2- 172.21.15.40:0/1360975235 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7efc15af40 0x7f7efc0cb0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.504+0000 7f7f01559640 1 -- 172.21.15.40:0/1360975235 >> v2:172.21.15.86:3300/0 conn(0x7f7efc0cb5f0 msgr2=0x7f7efc0714f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:49.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.504+0000 7f7f01559640 1 --2- 172.21.15.40:0/1360975235 >> v2:172.21.15.86:3300/0 conn(0x7f7efc0cb5f0 0x7f7efc0714f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.504+0000 7f7f01559640 1 -- 172.21.15.40:0/1360975235 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc06d870 con 0x7f7efc071a30 2023-10-16T08:38:49.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.504+0000 7f7efbfff640 1 --2- 172.21.15.40:0/1360975235 >> v2:172.21.15.86:3300/0 conn(0x7f7efc0cb5f0 0x7f7efc0714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:49.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f01559640 1 --2- 172.21.15.40:0/1360975235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7efc071a30 0x7f7efc15e890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7efc06ce40 tx=0x7f7eec01c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8dd7602036a9184 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:49.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7efb7fe640 1 -- 172.21.15.40:0/1360975235 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec0ab600 con 0x7f7efc071a30 2023-10-16T08:38:49.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7efb7fe640 1 -- 172.21.15.40:0/1360975235 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eec0a7050 con 0x7f7efc071a30 2023-10-16T08:38:49.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7efb7fe640 1 -- 172.21.15.40:0/1360975235 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eec0ab980 con 0x7f7efc071a30 2023-10-16T08:38:49.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f02fbc640 1 -- 172.21.15.40:0/1360975235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7efc071a30 msgr2=0x7f7efc15e890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:49.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f02fbc640 1 --2- 172.21.15.40:0/1360975235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7efc071a30 0x7f7efc15e890 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7efc06ce40 tx=0x7f7eec01c180 comp rx=0 tx=0).stop 2023-10-16T08:38:49.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f02fbc640 1 -- 172.21.15.40:0/1360975235 shutdown_connections 2023-10-16T08:38:49.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f02fbc640 1 --2- 172.21.15.40:0/1360975235 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7efc071a30 0x7f7efc15e890 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f02fbc640 1 --2- 172.21.15.40:0/1360975235 >> v2:172.21.15.86:3300/0 conn(0x7f7efc0cb5f0 0x7f7efc0714f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f02fbc640 1 --2- 172.21.15.40:0/1360975235 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7efc15af40 0x7f7efc0cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f02fbc640 1 -- 172.21.15.40:0/1360975235 >> 172.21.15.40:0/1360975235 conn(0x7f7efc0b0be0 msgr2=0x7f7efc0b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:49.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f02fbc640 1 -- 172.21.15.40:0/1360975235 shutdown_connections 2023-10-16T08:38:49.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f02fbc640 1 -- 172.21.15.40:0/1360975235 wait complete. 2023-10-16T08:38:49.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f02fbc640 1 Processor -- start 2023-10-16T08:38:49.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f02fbc640 1 -- start start 2023-10-16T08:38:49.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f02fbc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7efc071a30 0x7f7efc155230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:49.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f02fbc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7efc0cb5f0 0x7f7efc0af7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:49.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f02fbc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7efc15af40 0x7f7efc0afce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:49.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f02fbc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc15c3a0 con 0x7f7efc0cb5f0 2023-10-16T08:38:49.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f02fbc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc15b7f0 con 0x7f7efc15af40 2023-10-16T08:38:49.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f02fbc640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7efc15aa10 con 0x7f7efc071a30 2023-10-16T08:38:49.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7efbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7efc0cb5f0 0x7f7efc0af7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:49.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7efbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7efc0cb5f0 0x7f7efc0af7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58150/0 (socket says 172.21.15.40:58150) 2023-10-16T08:38:49.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7efbfff640 1 -- 172.21.15.40:0/2882830714 learned_addr learned my addr 172.21.15.40:0/2882830714 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:49.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f01559640 1 --2- 172.21.15.40:0/2882830714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7efc15af40 0x7f7efc0afce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:49.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f00d58640 1 --2- 172.21.15.40:0/2882830714 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7efc071a30 0x7f7efc155230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:49.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7efbfff640 1 -- 172.21.15.40:0/2882830714 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7efc071a30 msgr2=0x7f7efc155230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:49.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7efbfff640 1 --2- 172.21.15.40:0/2882830714 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7efc071a30 0x7f7efc155230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7efbfff640 1 -- 172.21.15.40:0/2882830714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7efc15af40 msgr2=0x7f7efc0afce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:49.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7efbfff640 1 --2- 172.21.15.40:0/2882830714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7efc15af40 0x7f7efc0afce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7efbfff640 1 -- 172.21.15.40:0/2882830714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eec0094d0 con 0x7f7efc0cb5f0 2023-10-16T08:38:49.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7efbfff640 1 --2- 172.21.15.40:0/2882830714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7efc0cb5f0 0x7f7efc0af7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7ee400c7c0 tx=0x7f7ee400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:49.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7ef97fa640 1 -- 172.21.15.40:0/2882830714 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee40079d0 con 0x7f7efc0cb5f0 2023-10-16T08:38:49.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7ef97fa640 1 -- 172.21.15.40:0/2882830714 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee4007b30 con 0x7f7efc0cb5f0 2023-10-16T08:38:49.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f02fbc640 1 -- 172.21.15.40:0/2882830714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7efc155770 con 0x7f7efc0cb5f0 2023-10-16T08:38:49.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7ef97fa640 1 -- 172.21.15.40:0/2882830714 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee400ce70 con 0x7f7efc0cb5f0 2023-10-16T08:38:49.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.508+0000 7f7f02fbc640 1 -- 172.21.15.40:0/2882830714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7efc15bce0 con 0x7f7efc0cb5f0 2023-10-16T08:38:49.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.512+0000 7f7f02fbc640 1 -- 172.21.15.40:0/2882830714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7efc15c4e0 con 0x7f7efc0cb5f0 2023-10-16T08:38:49.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.512+0000 7f7ef97fa640 1 -- 172.21.15.40:0/2882830714 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7ee40113f0 con 0x7f7efc0cb5f0 2023-10-16T08:38:49.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.512+0000 7f7ef97fa640 1 --2- 172.21.15.40:0/2882830714 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7edc0664a0 0x7f7edc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:49.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.512+0000 7f7ef97fa640 1 -- 172.21.15.40:0/2882830714 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7ee4083e20 con 0x7f7efc0cb5f0 2023-10-16T08:38:49.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.512+0000 7f7f00d58640 1 --2- 172.21.15.40:0/2882830714 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7edc0664a0 0x7f7edc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:49.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.512+0000 7f7f00d58640 1 --2- 172.21.15.40:0/2882830714 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7edc0664a0 0x7f7edc068960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40070b0 tx=0x7f7ef4007040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:49.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.516+0000 7f7ef97fa640 1 -- 172.21.15.40:0/2882830714 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f7efc15c4e0 con 0x7f7efc0cb5f0 2023-10-16T08:38:49.578 INFO:tasks.ceph.mds.c.smithi040.stderr:2023-10-16T08:38:49.572+0000 7f5d8abb6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T08:38:49.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.712+0000 7f7f02fbc640 1 -- 172.21.15.40:0/2882830714 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f7efc15c4e0 con 0x7f7efc0cb5f0 2023-10-16T08:38:49.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.716+0000 7f7ef97fa640 1 -- 172.21.15.40:0/2882830714 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+164430 (secure 0 0 0) 0x7f7efc15c4e0 con 0x7f7efc0cb5f0 2023-10-16T08:38:49.724 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:49.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.720+0000 7f7f02fbc640 1 -- 172.21.15.40:0/2882830714 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7edc0664a0 msgr2=0x7f7edc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:49.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.720+0000 7f7f02fbc640 1 --2- 172.21.15.40:0/2882830714 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7edc0664a0 0x7f7edc068960 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7ef40070b0 tx=0x7f7ef4007040 comp rx=0 tx=0).stop 2023-10-16T08:38:49.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.720+0000 7f7f02fbc640 1 -- 172.21.15.40:0/2882830714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7efc0cb5f0 msgr2=0x7f7efc0af7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:49.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.720+0000 7f7f02fbc640 1 --2- 172.21.15.40:0/2882830714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7efc0cb5f0 0x7f7efc0af7a0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f7ee400c7c0 tx=0x7f7ee400cc80 comp rx=0 tx=0).stop 2023-10-16T08:38:49.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.720+0000 7f7f02fbc640 1 -- 172.21.15.40:0/2882830714 shutdown_connections 2023-10-16T08:38:49.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.720+0000 7f7f02fbc640 1 --2- 172.21.15.40:0/2882830714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7efc15af40 0x7f7efc0afce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.720+0000 7f7f02fbc640 1 --2- 172.21.15.40:0/2882830714 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7edc0664a0 0x7f7edc068960 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.720+0000 7f7f02fbc640 1 --2- 172.21.15.40:0/2882830714 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7efc0cb5f0 0x7f7efc0af7a0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.720+0000 7f7f02fbc640 1 --2- 172.21.15.40:0/2882830714 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7efc071a30 0x7f7efc155230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.720+0000 7f7f02fbc640 1 -- 172.21.15.40:0/2882830714 >> 172.21.15.40:0/2882830714 conn(0x7f7efc0b0be0 msgr2=0x7f7efc067cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:49.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.720+0000 7f7f02fbc640 1 -- 172.21.15.40:0/2882830714 shutdown_connections 2023-10-16T08:38:49.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.720+0000 7f7f02fbc640 1 -- 172.21.15.40:0/2882830714 wait complete. 2023-10-16T08:38:49.742 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":5,"active_gid":4111,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1188970899},{"type":"v1","addr":"172.21.15.86:6833","nonce":1188970899}]},"active_addr":"172.21.15.86:6833/1188970899","active_change":"2023-10-16T08:38:24.920708+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4101,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.3","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":1036780522}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":3068217938}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":1660391658}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":2418735977}]}]} 2023-10-16T08:38:49.747 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-16T08:38:49.747 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-16T08:38:49.748 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:38:49.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.864+0000 7fb58775e640 1 Processor -- start 2023-10-16T08:38:49.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.864+0000 7fb58775e640 1 -- start start 2023-10-16T08:38:49.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.868+0000 7fb58775e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb58015d560 0x7fb58006b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:49.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.868+0000 7fb58775e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5800ad820 0x7fb5800c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:49.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.868+0000 7fb58775e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5800cf480 con 0x7fb58015d180 2023-10-16T08:38:49.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.868+0000 7fb58775e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5800cf760 con 0x7fb58015d560 2023-10-16T08:38:49.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.868+0000 7fb58775e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5800c8fb0 con 0x7fb5800ad820 2023-10-16T08:38:49.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.868+0000 7fb584cf9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb58015d560 0x7fb58006b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:49.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.868+0000 7fb584cf9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb58015d560 0x7fb58006b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58154/0 (socket says 172.21.15.40:58154) 2023-10-16T08:38:49.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.868+0000 7fb584cf9640 1 -- 172.21.15.40:0/2983980165 learned_addr learned my addr 172.21.15.40:0/2983980165 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:49.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.868+0000 7fb5854fa640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb58015d180 0x7fb5800c9570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58324/0 (socket says 172.21.15.40:58324) 2023-10-16T08:38:49.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.868+0000 7fb584cf9640 1 -- 172.21.15.40:0/2983980165 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5800ad820 msgr2=0x7fb5800c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:49.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.868+0000 7fb584cf9640 1 --2- 172.21.15.40:0/2983980165 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5800ad820 0x7fb5800c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.868+0000 7fb584cf9640 1 -- 172.21.15.40:0/2983980165 >> v1:172.21.15.40:6789/0 conn(0x7fb58015d180 legacy=0x7fb5800c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:49.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.868+0000 7fb584cf9640 1 -- 172.21.15.40:0/2983980165 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5800c9290 con 0x7fb58015d560 2023-10-16T08:38:49.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.868+0000 7fb584cf9640 1 --2- 172.21.15.40:0/2983980165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb58015d560 0x7fb58006b7e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00bb10 tx=0x7fb57c01eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a4ddbbbd1f4d01b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:49.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.868+0000 7fb577fff640 1 -- 172.21.15.40:0/2983980165 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c0a7c70 con 0x7fb58015d560 2023-10-16T08:38:49.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.868+0000 7fb577fff640 1 -- 172.21.15.40:0/2983980165 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb57c0a7dd0 con 0x7fb58015d560 2023-10-16T08:38:49.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.868+0000 7fb577fff640 1 -- 172.21.15.40:0/2983980165 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb57c0af660 con 0x7fb58015d560 2023-10-16T08:38:49.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb58775e640 1 -- 172.21.15.40:0/2983980165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb58015d560 msgr2=0x7fb58006b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:49.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb58775e640 1 --2- 172.21.15.40:0/2983980165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb58015d560 0x7fb58006b7e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb57c00bb10 tx=0x7fb57c01eb80 comp rx=0 tx=0).stop 2023-10-16T08:38:49.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb58775e640 1 -- 172.21.15.40:0/2983980165 shutdown_connections 2023-10-16T08:38:49.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb58775e640 1 --2- 172.21.15.40:0/2983980165 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb5800ad820 0x7fb5800c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb58775e640 1 --2- 172.21.15.40:0/2983980165 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb58015d560 0x7fb58006b7e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb58775e640 1 -- 172.21.15.40:0/2983980165 >> 172.21.15.40:0/2983980165 conn(0x7fb5800ce960 msgr2=0x7fb5800ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:49.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb58775e640 1 -- 172.21.15.40:0/2983980165 shutdown_connections 2023-10-16T08:38:49.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb58775e640 1 -- 172.21.15.40:0/2983980165 wait complete. 2023-10-16T08:38:49.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb58775e640 1 Processor -- start 2023-10-16T08:38:49.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb58775e640 1 -- start start 2023-10-16T08:38:49.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb58775e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5800ad820 0x7fb5800c7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:49.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb58775e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb58015d180 0x7fb5800af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:49.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb58775e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb58015d560 0x7fb5800c5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:49.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb58775e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5800afe00 con 0x7fb5800ad820 2023-10-16T08:38:49.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb58775e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58015bca0 con 0x7fb58015d180 2023-10-16T08:38:49.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb58775e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb58015bf40 con 0x7fb58015d560 2023-10-16T08:38:49.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb5854fa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5800ad820 0x7fb5800c7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:49.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb5854fa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5800ad820 0x7fb5800c7bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58164/0 (socket says 172.21.15.40:58164) 2023-10-16T08:38:49.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb5854fa640 1 -- 172.21.15.40:0/1426645510 learned_addr learned my addr 172.21.15.40:0/1426645510 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:49.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb5854fa640 1 -- 172.21.15.40:0/1426645510 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb58015d560 msgr2=0x7fb5800c5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:49.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb585cfb640 1 --2- 172.21.15.40:0/1426645510 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb58015d560 0x7fb5800c5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:49.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb584cf9640 1 --2- 172.21.15.40:0/1426645510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb58015d180 0x7fb5800af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:49.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb5854fa640 1 --2- 172.21.15.40:0/1426645510 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb58015d560 0x7fb5800c5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb5854fa640 1 -- 172.21.15.40:0/1426645510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb58015d180 msgr2=0x7fb5800af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:49.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb5854fa640 1 --2- 172.21.15.40:0/1426645510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb58015d180 0x7fb5800af8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:49.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb5854fa640 1 -- 172.21.15.40:0/1426645510 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb57c00b770 con 0x7fb5800ad820 2023-10-16T08:38:49.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb585cfb640 1 --2- 172.21.15.40:0/1426645510 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb58015d560 0x7fb5800c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:49.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb5854fa640 1 --2- 172.21.15.40:0/1426645510 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5800ad820 0x7fb5800c7bb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb57000c490 tx=0x7fb57000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:49.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb575ffb640 1 -- 172.21.15.40:0/1426645510 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb570007480 con 0x7fb5800ad820 2023-10-16T08:38:49.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb575ffb640 1 -- 172.21.15.40:0/1426645510 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5700075e0 con 0x7fb5800ad820 2023-10-16T08:38:49.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb58775e640 1 -- 172.21.15.40:0/1426645510 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5800c70d0 con 0x7fb5800ad820 2023-10-16T08:38:49.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb575ffb640 1 -- 172.21.15.40:0/1426645510 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb570004a10 con 0x7fb5800ad820 2023-10-16T08:38:49.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.872+0000 7fb58775e640 1 -- 172.21.15.40:0/1426645510 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5800c7690 con 0x7fb5800ad820 2023-10-16T08:38:49.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.876+0000 7fb58775e640 1 -- 172.21.15.40:0/1426645510 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5800c7880 con 0x7fb5800ad820 2023-10-16T08:38:49.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.876+0000 7fb575ffb640 1 -- 172.21.15.40:0/1426645510 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb570004b70 con 0x7fb5800ad820 2023-10-16T08:38:49.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.876+0000 7fb575ffb640 1 --2- 172.21.15.40:0/1426645510 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb5640664a0 0x7fb564068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:49.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.876+0000 7fb575ffb640 1 -- 172.21.15.40:0/1426645510 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb5700832c0 con 0x7fb5800ad820 2023-10-16T08:38:49.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.876+0000 7fb584cf9640 1 --2- 172.21.15.40:0/1426645510 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb5640664a0 0x7fb564068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:49.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.876+0000 7fb584cf9640 1 --2- 172.21.15.40:0/1426645510 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb5640664a0 0x7fb564068960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb57c0b1000 tx=0x7fb57c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:49.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:49.880+0000 7fb575ffb640 1 -- 172.21.15.40:0/1426645510 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fb570050740 con 0x7fb5800ad820 2023-10-16T08:38:50.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.036+0000 7fb58775e640 1 -- 172.21.15.40:0/1426645510 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb58004e9c0 con 0x7fb5800ad820 2023-10-16T08:38:50.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.036+0000 7fb575ffb640 1 -- 172.21.15.40:0/1426645510 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16445 (secure 0 0 0) 0x7fb570053d60 con 0x7fb5800ad820 2023-10-16T08:38:50.042 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:50.042 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:46.611098+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:38:50.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.040+0000 7fb58775e640 1 -- 172.21.15.40:0/1426645510 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb5640664a0 msgr2=0x7fb564068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.040+0000 7fb58775e640 1 --2- 172.21.15.40:0/1426645510 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb5640664a0 0x7fb564068960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fb57c0b1000 tx=0x7fb57c008040 comp rx=0 tx=0).stop 2023-10-16T08:38:50.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.040+0000 7fb58775e640 1 -- 172.21.15.40:0/1426645510 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5800ad820 msgr2=0x7fb5800c7bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.040+0000 7fb58775e640 1 --2- 172.21.15.40:0/1426645510 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5800ad820 0x7fb5800c7bb0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb57000c490 tx=0x7fb57000c950 comp rx=0 tx=0).stop 2023-10-16T08:38:50.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.040+0000 7fb58775e640 1 -- 172.21.15.40:0/1426645510 shutdown_connections 2023-10-16T08:38:50.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.040+0000 7fb58775e640 1 --2- 172.21.15.40:0/1426645510 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb58015d560 0x7fb5800c5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.040+0000 7fb58775e640 1 --2- 172.21.15.40:0/1426645510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb58015d180 0x7fb5800af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.040+0000 7fb58775e640 1 --2- 172.21.15.40:0/1426645510 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb5640664a0 0x7fb564068960 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.040+0000 7fb58775e640 1 --2- 172.21.15.40:0/1426645510 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb5800ad820 0x7fb5800c7bb0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.040+0000 7fb58775e640 1 -- 172.21.15.40:0/1426645510 >> 172.21.15.40:0/1426645510 conn(0x7fb5800ce960 msgr2=0x7fb58006b540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.040+0000 7fb58775e640 1 -- 172.21.15.40:0/1426645510 shutdown_connections 2023-10-16T08:38:50.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.040+0000 7fb58775e640 1 -- 172.21.15.40:0/1426645510 wait complete. 2023-10-16T08:38:50.059 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-16T08:38:50.059 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:38:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.164+0000 7fa454e4d640 1 Processor -- start 2023-10-16T08:38:50.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.164+0000 7fa454e4d640 1 -- start start 2023-10-16T08:38:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.168+0000 7fa454e4d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa45006bbe0 0x7fa45015e060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.168+0000 7fa454e4d640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa450069e90 0x7fa4500cd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.168+0000 7fa454e4d640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4500cfd30 con 0x7fa45015a970 2023-10-16T08:38:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.168+0000 7fa454e4d640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa45006a600 con 0x7fa450069e90 2023-10-16T08:38:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.168+0000 7fa454e4d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa45006a8e0 con 0x7fa45006bbe0 2023-10-16T08:38:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.168+0000 7fa44dd9b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa45006bbe0 0x7fa45015e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.168+0000 7fa44ed9d640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa450069e90 0x7fa4500cd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.168+0000 7fa44dd9b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa45006bbe0 0x7fa45015e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.40:54426/0 (socket says 172.21.15.40:54426) 2023-10-16T08:38:50.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.168+0000 7fa44dd9b640 1 -- 172.21.15.40:0/4038944442 learned_addr learned my addr 172.21.15.40:0/4038944442 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.168+0000 7fa44ed9d640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa450069e90 0x7fa4500cd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.40:53788/0 (socket says 172.21.15.40:53788) 2023-10-16T08:38:50.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.168+0000 7fa44e59c640 1 --1- 172.21.15.40:0/4038944442 >> v1:172.21.15.86:6789/0 conn(0x7fa45015a970 0x7fa45006b4d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.40:47170/0 (socket says 172.21.15.40:47170) 2023-10-16T08:38:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.168+0000 7fa44dd9b640 1 -- 172.21.15.40:0/4038944442 >> v2:172.21.15.86:3300/0 conn(0x7fa450069e90 msgr2=0x7fa4500cd130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.168+0000 7fa44dd9b640 1 --2- 172.21.15.40:0/4038944442 >> v2:172.21.15.86:3300/0 conn(0x7fa450069e90 0x7fa4500cd130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.168+0000 7fa44dd9b640 1 -- 172.21.15.40:0/4038944442 >> v1:172.21.15.86:6789/0 conn(0x7fa45015a970 legacy=0x7fa45006b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.168+0000 7fa44dd9b640 1 -- 172.21.15.40:0/4038944442 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa45006abc0 con 0x7fa45006bbe0 2023-10-16T08:38:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.168+0000 7fa44ed9d640 1 --2- 172.21.15.40:0/4038944442 >> v2:172.21.15.86:3300/0 conn(0x7fa450069e90 0x7fa4500cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:38:50.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.168+0000 7fa44dd9b640 1 --2- 172.21.15.40:0/4038944442 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa45006bbe0 0x7fa45015e060 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa440009870 tx=0x7fa44001c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e0a6ccffcb21b8ed server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa44d59a640 1 -- 172.21.15.40:0/4038944442 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4400a9620 con 0x7fa45006bbe0 2023-10-16T08:38:50.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa44d59a640 1 -- 172.21.15.40:0/4038944442 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4400a9780 con 0x7fa45006bbe0 2023-10-16T08:38:50.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa44d59a640 1 -- 172.21.15.40:0/4038944442 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4400a9b10 con 0x7fa45006bbe0 2023-10-16T08:38:50.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa454e4d640 1 -- 172.21.15.40:0/4038944442 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa45006bbe0 msgr2=0x7fa45015e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa454e4d640 1 --2- 172.21.15.40:0/4038944442 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa45006bbe0 0x7fa45015e060 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fa440009870 tx=0x7fa44001c540 comp rx=0 tx=0).stop 2023-10-16T08:38:50.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa454e4d640 1 -- 172.21.15.40:0/4038944442 shutdown_connections 2023-10-16T08:38:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa454e4d640 1 --2- 172.21.15.40:0/4038944442 >> v2:172.21.15.86:3300/0 conn(0x7fa450069e90 0x7fa4500cd130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa454e4d640 1 --2- 172.21.15.40:0/4038944442 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa45006bbe0 0x7fa45015e060 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa454e4d640 1 -- 172.21.15.40:0/4038944442 >> 172.21.15.40:0/4038944442 conn(0x7fa450071620 msgr2=0x7fa450071a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa454e4d640 1 -- 172.21.15.40:0/4038944442 shutdown_connections 2023-10-16T08:38:50.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa454e4d640 1 -- 172.21.15.40:0/4038944442 wait complete. 2023-10-16T08:38:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa454e4d640 1 Processor -- start 2023-10-16T08:38:50.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa454e4d640 1 -- start start 2023-10-16T08:38:50.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa454e4d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa450069e90 0x7fa45015f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa454e4d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa45006bbe0 0x7fa450153a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa454e4d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa45015a970 0x7fa450153f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa454e4d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500c7b90 con 0x7fa45015a970 2023-10-16T08:38:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa454e4d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500ac8e0 con 0x7fa45006bbe0 2023-10-16T08:38:50.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa454e4d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4500acea0 con 0x7fa450069e90 2023-10-16T08:38:50.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa44ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa45015a970 0x7fa450153f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa44ed9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa45015a970 0x7fa450153f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58166/0 (socket says 172.21.15.40:58166) 2023-10-16T08:38:50.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa44ed9d640 1 -- 172.21.15.40:0/3924040242 learned_addr learned my addr 172.21.15.40:0/3924040242 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa44e59c640 1 --2- 172.21.15.40:0/3924040242 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa450069e90 0x7fa45015f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa44dd9b640 1 --2- 172.21.15.40:0/3924040242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa45006bbe0 0x7fa450153a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa44ed9d640 1 -- 172.21.15.40:0/3924040242 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa450069e90 msgr2=0x7fa45015f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa44ed9d640 1 --2- 172.21.15.40:0/3924040242 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa450069e90 0x7fa45015f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa44ed9d640 1 -- 172.21.15.40:0/3924040242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa45006bbe0 msgr2=0x7fa450153a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa44ed9d640 1 --2- 172.21.15.40:0/3924040242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa45006bbe0 0x7fa450153a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa44ed9d640 1 -- 172.21.15.40:0/3924040242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa448009a70 con 0x7fa45015a970 2023-10-16T08:38:50.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa44ed9d640 1 --2- 172.21.15.40:0/3924040242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa45015a970 0x7fa450153f90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa44800efc0 tx=0x7fa448009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa446ffd640 1 -- 172.21.15.40:0/3924040242 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa448009d70 con 0x7fa45015a970 2023-10-16T08:38:50.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa446ffd640 1 -- 172.21.15.40:0/3924040242 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa44800f040 con 0x7fa45015a970 2023-10-16T08:38:50.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa454e4d640 1 -- 172.21.15.40:0/3924040242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4400094d0 con 0x7fa45015a970 2023-10-16T08:38:50.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa446ffd640 1 -- 172.21.15.40:0/3924040242 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4480039a0 con 0x7fa45015a970 2023-10-16T08:38:50.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.172+0000 7fa454e4d640 1 -- 172.21.15.40:0/3924040242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4500ad650 con 0x7fa45015a970 2023-10-16T08:38:50.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.176+0000 7fa454e4d640 1 -- 172.21.15.40:0/3924040242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa450067a20 con 0x7fa45015a970 2023-10-16T08:38:50.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.176+0000 7fa446ffd640 1 -- 172.21.15.40:0/3924040242 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa448014480 con 0x7fa45015a970 2023-10-16T08:38:50.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.176+0000 7fa446ffd640 1 --2- 172.21.15.40:0/3924040242 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa4300664a0 0x7fa430068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.176+0000 7fa446ffd640 1 -- 172.21.15.40:0/3924040242 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa448084bb0 con 0x7fa45015a970 2023-10-16T08:38:50.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.176+0000 7fa44e59c640 1 --2- 172.21.15.40:0/3924040242 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa4300664a0 0x7fa430068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.176+0000 7fa44e59c640 1 --2- 172.21.15.40:0/3924040242 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa4300664a0 0x7fa430068960 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa438000f70 tx=0x7fa438007cb0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.180+0000 7fa446ffd640 1 -- 172.21.15.40:0/3924040242 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fa448051fb0 con 0x7fa45015a970 2023-10-16T08:38:50.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.336+0000 7fa454e4d640 1 -- 172.21.15.40:0/3924040242 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa45006c020 con 0x7fa45015a970 2023-10-16T08:38:50.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.340+0000 7fa446ffd640 1 -- 172.21.15.40:0/3924040242 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16445 (secure 0 0 0) 0x7fa4480555d0 con 0x7fa45015a970 2023-10-16T08:38:50.344 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:50.345 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:46.611098+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:38:50.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.340+0000 7fa454e4d640 1 -- 172.21.15.40:0/3924040242 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa4300664a0 msgr2=0x7fa430068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.340+0000 7fa454e4d640 1 --2- 172.21.15.40:0/3924040242 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa4300664a0 0x7fa430068960 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa438000f70 tx=0x7fa438007cb0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.340+0000 7fa454e4d640 1 -- 172.21.15.40:0/3924040242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa45015a970 msgr2=0x7fa450153f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.340+0000 7fa454e4d640 1 --2- 172.21.15.40:0/3924040242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa45015a970 0x7fa450153f90 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa44800efc0 tx=0x7fa448009f90 comp rx=0 tx=0).stop 2023-10-16T08:38:50.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.340+0000 7fa454e4d640 1 -- 172.21.15.40:0/3924040242 shutdown_connections 2023-10-16T08:38:50.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.340+0000 7fa454e4d640 1 --2- 172.21.15.40:0/3924040242 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa4300664a0 0x7fa430068960 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.340+0000 7fa454e4d640 1 --2- 172.21.15.40:0/3924040242 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa45015a970 0x7fa450153f90 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.340+0000 7fa454e4d640 1 --2- 172.21.15.40:0/3924040242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa45006bbe0 0x7fa450153a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.340+0000 7fa454e4d640 1 --2- 172.21.15.40:0/3924040242 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa450069e90 0x7fa45015f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.340+0000 7fa454e4d640 1 -- 172.21.15.40:0/3924040242 >> 172.21.15.40:0/3924040242 conn(0x7fa450071620 msgr2=0x7fa450156290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.340+0000 7fa454e4d640 1 -- 172.21.15.40:0/3924040242 shutdown_connections 2023-10-16T08:38:50.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.340+0000 7fa454e4d640 1 -- 172.21.15.40:0/3924040242 wait complete. 2023-10-16T08:38:50.362 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-16T08:38:50.363 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-16T08:38:50.363 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-16T08:38:50.364 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-16T08:38:50.364 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-16T08:38:50.365 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-16T08:38:50.365 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-16T08:38:50.366 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-16T08:38:50.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.544+0000 7fb3d6199640 1 Processor -- start 2023-10-16T08:38:50.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.544+0000 7fb3d6199640 1 -- start start 2023-10-16T08:38:50.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.544+0000 7fb3d6199640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3d00cde30 0x7fb3d00c9320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.544+0000 7fb3d6199640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3d00c9860 0x7fb3d016d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.544+0000 7fb3d6199640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb3d016db40 0x7fb3d00b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.544+0000 7fb3d6199640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d00caf50 con 0x7fb3d00c9860 2023-10-16T08:38:50.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.544+0000 7fb3d6199640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3d016dfb0 con 0x7fb3d016db40 2023-10-16T08:38:50.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.544+0000 7fb3d6199640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d00ca460 con 0x7fb3d00cde30 2023-10-16T08:38:50.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.548+0000 7fb3ceffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3d00c9860 0x7fb3d016d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.548+0000 7fb3ceffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb3d00c9860 0x7fb3d016d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58168/0 (socket says 172.21.15.40:58168) 2023-10-16T08:38:50.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.548+0000 7fb3ceffd640 1 -- 172.21.15.40:0/1039241450 learned_addr learned my addr 172.21.15.40:0/1039241450 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.548+0000 7fb3cffff640 1 --2- 172.21.15.40:0/1039241450 >> v2:172.21.15.86:3300/0 conn(0x7fb3d016db40 0x7fb3d00b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.548+0000 7fb3cf7fe640 1 --2- 172.21.15.40:0/1039241450 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3d00cde30 0x7fb3d00c9320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.548+0000 7fb3ceffd640 1 -- 172.21.15.40:0/1039241450 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3d00cde30 msgr2=0x7fb3d00c9320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.548+0000 7fb3ceffd640 1 --2- 172.21.15.40:0/1039241450 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3d00cde30 0x7fb3d00c9320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.548+0000 7fb3ceffd640 1 -- 172.21.15.40:0/1039241450 >> v2:172.21.15.86:3300/0 conn(0x7fb3d016db40 msgr2=0x7fb3d00b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.548+0000 7fb3ceffd640 1 --2- 172.21.15.40:0/1039241450 >> v2:172.21.15.86:3300/0 conn(0x7fb3d016db40 0x7fb3d00b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.548+0000 7fb3ceffd640 1 -- 172.21.15.40:0/1039241450 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d00ca990 con 0x7fb3d00c9860 2023-10-16T08:38:50.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.548+0000 7fb3ceffd640 1 --2- 172.21.15.40:0/1039241450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d00c9860 0x7fb3d016d600 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb3d008aec0 tx=0x7fb3c801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c3ff27543905039 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7fb3ce7fc640 1 -- 172.21.15.40:0/1039241450 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c80a5730 con 0x7fb3d00c9860 2023-10-16T08:38:50.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7fb3ce7fc640 1 -- 172.21.15.40:0/1039241450 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c80a5890 con 0x7fb3d00c9860 2023-10-16T08:38:50.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7fb3ce7fc640 1 -- 172.21.15.40:0/1039241450 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c80a5c20 con 0x7fb3d00c9860 2023-10-16T08:38:50.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7fb3d6199640 1 -- 172.21.15.40:0/1039241450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d00c9860 msgr2=0x7fb3d016d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7fb3d6199640 1 --2- 172.21.15.40:0/1039241450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d00c9860 0x7fb3d016d600 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb3d008aec0 tx=0x7fb3c801c9a0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f644f4640 1 Processor -- start 2023-10-16T08:38:50.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f644f4640 1 -- start start 2023-10-16T08:38:50.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f644f4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0f5c0cd410 0x7f0f5c0c8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f644f4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5c0cc510 con 0x7f0f5c0ccf60 2023-10-16T08:38:50.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f644f4640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5c0ca610 con 0x7f0f5c0c1bf0 2023-10-16T08:38:50.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f644f4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5c0ca8f0 con 0x7f0f5c0cd410 2023-10-16T08:38:50.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f61a8f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0f5c0ccf60 0x7f0f5c06c410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58338/0 (socket says 172.21.15.40:58338) 2023-10-16T08:38:50.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f61a8f640 1 -- 172.21.15.40:0/3831125077 learned_addr learned my addr 172.21.15.40:0/3831125077 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:50.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f62a91640 1 --2- 172.21.15.40:0/3831125077 >> v2:172.21.15.40:3300/0 conn(0x7f0f5c0cd410 0x7f0f5c0c8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f6128e640 1 -- 172.21.15.40:0/3831125077 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 14881999 0 0) 0x7f0f5c0cc510 con 0x7f0f5c0ccf60 2023-10-16T08:38:50.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f6128e640 1 -- 172.21.15.40:0/3831125077 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0f48003530 con 0x7f0f5c0ccf60 2023-10-16T08:38:50.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f6128e640 1 -- 172.21.15.40:0/3831125077 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2977479218 0 0) 0x7f0f48003530 con 0x7f0f5c0ccf60 2023-10-16T08:38:50.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f6128e640 1 -- 172.21.15.40:0/3831125077 >> v2:172.21.15.40:3300/0 conn(0x7f0f5c0cd410 msgr2=0x7f0f5c0c8c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f6128e640 1 --2- 172.21.15.40:0/3831125077 >> v2:172.21.15.40:3300/0 conn(0x7f0f5c0cd410 0x7f0f5c0c8c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f6128e640 1 -- 172.21.15.40:0/3831125077 >> v1:172.21.15.86:6789/0 conn(0x7f0f5c0c1bf0 legacy=0x7f0f5c0cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f6128e640 1 -- 172.21.15.40:0/3831125077 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c0cabd0 con 0x7f0f5c0ccf60 2023-10-16T08:38:50.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f6128e640 1 -- 172.21.15.40:0/3831125077 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f0f54001a00 con 0x7f0f5c0ccf60 2023-10-16T08:38:50.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f6128e640 1 -- 172.21.15.40:0/3831125077 >> v1:172.21.15.40:6789/0 conn(0x7f0f5c0ccf60 legacy=0x7f0f5c06c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f6128e640 1 --2- 172.21.15.40:0/3831125077 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f480039f0 0x7f0f48003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f6128e640 1 --2- 172.21.15.40:0/3831125077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f480043a0 0x7f0f48004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f6128e640 1 --2- 172.21.15.40:0/3831125077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f48005070 0x7f0f4800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f6128e640 1 -- 172.21.15.40:0/3831125077 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c0ca8f0 con 0x7f0f48005070 2023-10-16T08:38:50.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f6128e640 1 -- 172.21.15.40:0/3831125077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4800bfb0 con 0x7f0f480043a0 2023-10-16T08:38:50.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f6128e640 1 -- 172.21.15.40:0/3831125077 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f4800c170 con 0x7f0f480039f0 2023-10-16T08:38:50.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f62a91640 1 --2- 172.21.15.40:0/3831125077 >> v2:172.21.15.40:3300/0 conn(0x7f0f5c0cd410 0x7f0f5c0c8c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:38:50.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f62290640 1 --2- 172.21.15.40:0/3831125077 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f480039f0 0x7f0f48003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f61a8f640 1 --2- 172.21.15.40:0/3831125077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f480043a0 0x7f0f48004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f62a91640 1 --2- 172.21.15.40:0/3831125077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f48005070 0x7f0f4800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f408d7c6640 1 Processor -- start 2023-10-16T08:38:50.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7fb3d6199640 1 -- 172.21.15.40:0/1039241450 shutdown_connections 2023-10-16T08:38:50.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7fb3d6199640 1 --2- 172.21.15.40:0/1039241450 >> v2:172.21.15.86:3300/0 conn(0x7fb3d016db40 0x7fb3d00b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7fb3d6199640 1 --2- 172.21.15.40:0/1039241450 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d00c9860 0x7fb3d016d600 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7fb3d6199640 1 --2- 172.21.15.40:0/1039241450 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3d00cde30 0x7fb3d00c9320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7fb3d6199640 1 -- 172.21.15.40:0/1039241450 >> 172.21.15.40:0/1039241450 conn(0x7fb3d00cb090 msgr2=0x7fb3d00cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f408d7c6640 1 -- start start 2023-10-16T08:38:50.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f62290640 1 -- 172.21.15.40:0/3831125077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f480043a0 msgr2=0x7f0f48004870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f62290640 1 --2- 172.21.15.40:0/3831125077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f480043a0 0x7f0f48004870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.552+0000 7f0f62290640 1 -- 172.21.15.40:0/3831125077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f48005070 msgr2=0x7f0f4800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7f408d7c6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40880acf80 0x7f4088161230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7fb3d6199640 1 -- 172.21.15.40:0/1039241450 shutdown_connections 2023-10-16T08:38:50.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7f408d7c6640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f408815df80 con 0x7f408815d780 2023-10-16T08:38:50.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7f408d7c6640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40880aebf0 con 0x7f40880cfad0 2023-10-16T08:38:50.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7f408d7c6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408815dca0 con 0x7f40880acf80 2023-10-16T08:38:50.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7fb3d6199640 1 -- 172.21.15.40:0/1039241450 wait complete. 2023-10-16T08:38:50.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7fd242dcb640 1 Processor -- start 2023-10-16T08:38:50.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7f40877fe640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f40880cfad0 0x7f408815d070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.40:47194/0 (socket says 172.21.15.40:47194) 2023-10-16T08:38:50.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7f40877fe640 1 -- 172.21.15.40:0/3800233089 learned_addr learned my addr 172.21.15.40:0/3800233089 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:50.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7f4087fff640 1 --2- 172.21.15.40:0/3800233089 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40880acf80 0x7f4088161230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7fd242dcb640 1 -- start start 2023-10-16T08:38:50.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7f408cfc5640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f408815d780 0x7f408806c330 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58340/0 (socket says 172.21.15.40:58340) 2023-10-16T08:38:50.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7fb3d6199640 1 Processor -- start 2023-10-16T08:38:50.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7fb3d6199640 1 -- start start 2023-10-16T08:38:50.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7f4087fff640 1 -- 172.21.15.40:0/3800233089 >> v1:172.21.15.86:6789/0 conn(0x7f40880cfad0 legacy=0x7f408815d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7f0f62290640 1 --2- 172.21.15.40:0/3831125077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f48005070 0x7f0f4800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7f0f62290640 1 -- 172.21.15.40:0/3831125077 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c0cabd0 con 0x7f0f480039f0 2023-10-16T08:38:50.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7f0f62a91640 1 --2- 172.21.15.40:0/3831125077 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f48005070 0x7f0f4800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:38:50.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7fd242dcb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd23c160510 0x7fd23c0d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7f0f62a91640 1 -- 172.21.15.40:0/3831125077 reap_dead start 2023-10-16T08:38:50.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7fd242dcb640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd23c0cddc0 0x7fd23c0cfc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7fd242dcb640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd23c0d3300 con 0x7fd23c0d0140 2023-10-16T08:38:50.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7f0f62290640 1 --2- 172.21.15.40:0/3831125077 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f480039f0 0x7f0f48003dd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0f58001860 tx=0x7f0f5801c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3fc6942b0d782c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7fd240b67640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd23c160510 0x7fd23c0d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7fd23bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd23c0cddc0 0x7fd23c0cfc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7fd240b67640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd23c160510 0x7fd23c0d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.40:54476/0 (socket says 172.21.15.40:54476) 2023-10-16T08:38:50.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7fd240b67640 1 -- 172.21.15.40:0/2935191780 learned_addr learned my addr 172.21.15.40:0/2935191780 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7f0f6128e640 1 -- 172.21.15.40:0/3831125077 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f580052d0 con 0x7f0f480039f0 2023-10-16T08:38:50.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7f0f6128e640 1 -- 172.21.15.40:0/3831125077 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f58005430 con 0x7f0f480039f0 2023-10-16T08:38:50.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7f0f6128e640 1 -- 172.21.15.40:0/3831125077 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f58003820 con 0x7f0f480039f0 2023-10-16T08:38:50.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7fd242dcb640 1 -- 172.21.15.40:0/2935191780 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd23c0d05d0 con 0x7fd23c0cddc0 2023-10-16T08:38:50.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7f4087fff640 1 -- 172.21.15.40:0/3800233089 >> v1:172.21.15.40:6789/0 conn(0x7f408815d780 legacy=0x7f408806c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7f4086ffd640 1 -- 172.21.15.40:0/3800233089 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2040639325 0 0) 0x7f40880aebf0 con 0x7f40880cfad0 2023-10-16T08:38:50.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.556+0000 7f4087fff640 1 -- 172.21.15.40:0/3800233089 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4088160750 con 0x7f40880acf80 2023-10-16T08:38:50.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fd242dcb640 1 -- 172.21.15.40:0/2935191780 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c0d35e0 con 0x7fd23c160510 2023-10-16T08:38:50.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3d6199640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d00cde30 0x7fb3d00b2140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3d6199640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d016db40 0x7fb3d00c2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3d6199640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3d00ae080 0x7fb3d00c3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3d6199640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0171550 con 0x7fb3d016db40 2023-10-16T08:38:50.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3d6199640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d0085ba0 con 0x7fb3d00cde30 2023-10-16T08:38:50.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3d6199640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3d00b2680 con 0x7fb3d00ae080 2023-10-16T08:38:50.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fd23b7fe640 1 -- 172.21.15.40:0/2935191780 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3058911322 0 0) 0x7fd23c0d3300 con 0x7fd23c0d0140 2023-10-16T08:38:50.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fd23b7fe640 1 -- 172.21.15.40:0/2935191780 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd22c003530 con 0x7fd23c0d0140 2023-10-16T08:38:50.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fd23bfff640 1 -- 172.21.15.40:0/2935191780 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd23c160510 msgr2=0x7fd23c0d2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fd23bfff640 1 --2- 172.21.15.40:0/2935191780 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd23c160510 0x7fd23c0d2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fd23bfff640 1 -- 172.21.15.40:0/2935191780 >> v1:172.21.15.86:6789/0 conn(0x7fd23c0d0140 legacy=0x7fd23c0b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fd23bfff640 1 -- 172.21.15.40:0/2935191780 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd23c0d38c0 con 0x7fd23c0cddc0 2023-10-16T08:38:50.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3cffff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3d00ae080 0x7fb3d00c3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7f0f644f4640 1 -- 172.21.15.40:0/3831125077 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f480039f0 msgr2=0x7f0f48003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7f0f644f4640 1 --2- 172.21.15.40:0/3831125077 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f480039f0 0x7f0f48003dd0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0f58001860 tx=0x7f0f5801c520 comp rx=0 tx=0).stop 2023-10-16T08:38:50.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3cffff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3d00ae080 0x7fb3d00c3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.40:54482/0 (socket says 172.21.15.40:54482) 2023-10-16T08:38:50.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3cffff640 1 -- 172.21.15.40:0/3698858642 learned_addr learned my addr 172.21.15.40:0/3698858642 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7f0f644f4640 1 -- 172.21.15.40:0/3831125077 shutdown_connections 2023-10-16T08:38:50.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7f0f644f4640 1 --2- 172.21.15.40:0/3831125077 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f480039f0 0x7f0f48003dd0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7f0f644f4640 1 -- 172.21.15.40:0/3831125077 >> 172.21.15.40:0/3831125077 conn(0x7f0f5c0cb090 msgr2=0x7f0f5c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7f0f644f4640 1 -- 172.21.15.40:0/3831125077 shutdown_connections 2023-10-16T08:38:50.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7f0f644f4640 1 -- 172.21.15.40:0/3831125077 wait complete. 2023-10-16T08:38:50.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3cf7fe640 1 --2- 172.21.15.40:0/3698858642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d00cde30 0x7fb3d00b2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3ceffd640 1 --2- 172.21.15.40:0/3698858642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d016db40 0x7fb3d00c2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7f0f644f4640 1 Processor -- start 2023-10-16T08:38:50.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7f0f644f4640 1 -- start start 2023-10-16T08:38:50.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3cf7fe640 1 -- 172.21.15.40:0/3698858642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3d00ae080 msgr2=0x7fb3d00c3eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3cf7fe640 1 --2- 172.21.15.40:0/3698858642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3d00ae080 0x7fb3d00c3eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3cf7fe640 1 -- 172.21.15.40:0/3698858642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d016db40 msgr2=0x7fb3d00c2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3cf7fe640 1 --2- 172.21.15.40:0/3698858642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3d016db40 0x7fb3d00c2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3cf7fe640 1 -- 172.21.15.40:0/3698858642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3c8009b30 con 0x7fb3d00cde30 2023-10-16T08:38:50.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3cf7fe640 1 --2- 172.21.15.40:0/3698858642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d00cde30 0x7fb3d00b2140 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000d5f0 tx=0x7fb3c000dab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3bffff640 1 -- 172.21.15.40:0/3698858642 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c00125f0 con 0x7fb3d00cde30 2023-10-16T08:38:50.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3bffff640 1 -- 172.21.15.40:0/3698858642 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3c0012750 con 0x7fb3d00cde30 2023-10-16T08:38:50.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3d6199640 1 -- 172.21.15.40:0/3698858642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3d0170f90 con 0x7fb3d00cde30 2023-10-16T08:38:50.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7fb3d6199640 1 -- 172.21.15.40:0/3698858642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3d00c36b0 con 0x7fb3d00cde30 2023-10-16T08:38:50.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.560+0000 7f4087fff640 1 --2- 172.21.15.40:0/3800233089 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40880acf80 0x7f4088161230 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4080009dc0 tx=0x7f408806d250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97b5aad5d0f14b05 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7fd23bfff640 1 --2- 172.21.15.40:0/2935191780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd23c0cddc0 0x7fd23c0cfc00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd2300097c0 tx=0x7fd23001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44c3bdbd0aaddcc4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f4086ffd640 1 -- 172.21.15.40:0/3800233089 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40800a5a00 con 0x7f40880acf80 2023-10-16T08:38:50.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f4086ffd640 1 -- 172.21.15.40:0/3800233089 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40800a5b60 con 0x7f40880acf80 2023-10-16T08:38:50.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f4086ffd640 1 -- 172.21.15.40:0/3800233089 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40800ad440 con 0x7f40880acf80 2023-10-16T08:38:50.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7fb3d6199640 1 -- 172.21.15.40:0/3698858642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb3d00b09e0 con 0x7fb3d00cde30 2023-10-16T08:38:50.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f644f4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f480039f0 0x7f0f5c159270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f644f4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f5c0c4430 0x7f0f5c0c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f644f4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f5c0c7df0 0x7f0f5c1f9c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f644f4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c0c82a0 con 0x7f0f5c0c7df0 2023-10-16T08:38:50.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f644f4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c1597b0 con 0x7f0f480039f0 2023-10-16T08:38:50.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7fd23b7fe640 1 -- 172.21.15.40:0/2935191780 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2300ab670 con 0x7fd23c0cddc0 2023-10-16T08:38:50.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7fd23b7fe640 1 -- 172.21.15.40:0/2935191780 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2300ab7d0 con 0x7fd23c0cddc0 2023-10-16T08:38:50.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7fd23b7fe640 1 -- 172.21.15.40:0/2935191780 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2300abb60 con 0x7fd23c0cddc0 2023-10-16T08:38:50.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f644f4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c1faa10 con 0x7f0f5c0c4430 2023-10-16T08:38:50.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f61a8f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f5c0c4430 0x7f0f5c0c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f61a8f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f5c0c4430 0x7f0f5c0c78b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.40:54496/0 (socket says 172.21.15.40:54496) 2023-10-16T08:38:50.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f61a8f640 1 -- 172.21.15.40:0/1904429788 learned_addr learned my addr 172.21.15.40:0/1904429788 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f62290640 1 --2- 172.21.15.40:0/1904429788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f480039f0 0x7f0f5c159270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f62a91640 1 --2- 172.21.15.40:0/1904429788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f5c0c7df0 0x7f0f5c1f9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f61a8f640 1 -- 172.21.15.40:0/1904429788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f480039f0 msgr2=0x7f0f5c159270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f61a8f640 1 --2- 172.21.15.40:0/1904429788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f480039f0 0x7f0f5c159270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f61a8f640 1 -- 172.21.15.40:0/1904429788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f5c0c7df0 msgr2=0x7f0f5c1f9c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7fd242dcb640 1 -- 172.21.15.40:0/2935191780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd23c0cddc0 msgr2=0x7fd23c0cfc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7fd242dcb640 1 --2- 172.21.15.40:0/2935191780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd23c0cddc0 0x7fd23c0cfc00 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fd2300097c0 tx=0x7fd23001c450 comp rx=0 tx=0).stop 2023-10-16T08:38:50.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f408d7c6640 1 -- 172.21.15.40:0/3800233089 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40880acf80 msgr2=0x7f4088161230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f408d7c6640 1 --2- 172.21.15.40:0/3800233089 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40880acf80 0x7f4088161230 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4080009dc0 tx=0x7f408806d250 comp rx=0 tx=0).stop 2023-10-16T08:38:50.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f61a8f640 1 --2- 172.21.15.40:0/1904429788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f5c0c7df0 0x7f0f5c1f9c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f61a8f640 1 -- 172.21.15.40:0/1904429788 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c0c3670 con 0x7f0f5c0c4430 2023-10-16T08:38:50.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7fb3bffff640 1 -- 172.21.15.40:0/3698858642 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3c0012ae0 con 0x7fb3d00cde30 2023-10-16T08:38:50.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f62a91640 1 --2- 172.21.15.40:0/1904429788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f5c0c7df0 0x7f0f5c1f9c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:38:50.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f408d7c6640 1 -- 172.21.15.40:0/3800233089 shutdown_connections 2023-10-16T08:38:50.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f408d7c6640 1 --2- 172.21.15.40:0/3800233089 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40880acf80 0x7f4088161230 secure :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f4080009dc0 tx=0x7f408806d250 comp rx=0 tx=0).stop 2023-10-16T08:38:50.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f408d7c6640 1 -- 172.21.15.40:0/3800233089 >> 172.21.15.40:0/3800233089 conn(0x7f408806a290 msgr2=0x7f408806a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f61a8f640 1 --2- 172.21.15.40:0/1904429788 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f5c0c4430 0x7f0f5c0c78b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0f540079a0 tx=0x7f0f540225c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7fd242dcb640 1 -- 172.21.15.40:0/2935191780 shutdown_connections 2023-10-16T08:38:50.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7fd242dcb640 1 --2- 172.21.15.40:0/2935191780 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd23c0cddc0 0x7fd23c0cfc00 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7fd242dcb640 1 --2- 172.21.15.40:0/2935191780 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd23c160510 0x7fd23c0d2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7fd242dcb640 1 -- 172.21.15.40:0/2935191780 >> 172.21.15.40:0/2935191780 conn(0x7fd23c06a060 msgr2=0x7fd23c06a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7fd242dcb640 1 -- 172.21.15.40:0/2935191780 shutdown_connections 2023-10-16T08:38:50.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7fd242dcb640 1 -- 172.21.15.40:0/2935191780 wait complete. 2023-10-16T08:38:50.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f52ffd640 1 -- 172.21.15.40:0/1904429788 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f54026610 con 0x7f0f5c0c4430 2023-10-16T08:38:50.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f52ffd640 1 -- 172.21.15.40:0/1904429788 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f54026770 con 0x7f0f5c0c4430 2023-10-16T08:38:50.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f52ffd640 1 -- 172.21.15.40:0/1904429788 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f54026b20 con 0x7f0f5c0c4430 2023-10-16T08:38:50.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f644f4640 1 -- 172.21.15.40:0/1904429788 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5c0c3950 con 0x7f0f5c0c4430 2023-10-16T08:38:50.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f0f644f4640 1 -- 172.21.15.40:0/1904429788 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5c0c3f10 con 0x7f0f5c0c4430 2023-10-16T08:38:50.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7fd242dcb640 1 Processor -- start 2023-10-16T08:38:50.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f408d7c6640 1 -- 172.21.15.40:0/3800233089 shutdown_connections 2023-10-16T08:38:50.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7f408d7c6640 1 -- 172.21.15.40:0/3800233089 wait complete. 2023-10-16T08:38:50.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7fb3bffff640 1 -- 172.21.15.40:0/3698858642 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb3c0012c40 con 0x7fb3d00cde30 2023-10-16T08:38:50.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7fb3bffff640 1 --2- 172.21.15.40:0/3698858642 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb3b4066470 0x7fb3b4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7fb3bffff640 1 -- 172.21.15.40:0/3698858642 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3c0084dc0 con 0x7fb3d00cde30 2023-10-16T08:38:50.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fb3bffff640 1 --2- 172.21.15.40:0/3698858642 >> [v2:172.21.15.40:6800/2328483711,v1:172.21.15.40:6801/2328483711] conn(0x7fb3b406e610 0x7fb3b4070a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fb3bffff640 1 -- 172.21.15.40:0/3698858642 --> [v2:172.21.15.40:6800/2328483711,v1:172.21.15.40:6801/2328483711] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb3b40710e0 con 0x7fb3b406e610 2023-10-16T08:38:50.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fb3bffff640 1 -- 172.21.15.40:0/3698858642 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3c0085170 con 0x7fb3d00cde30 2023-10-16T08:38:50.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.564+0000 7fd242dcb640 1 -- start start 2023-10-16T08:38:50.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fb3ceffd640 1 --2- 172.21.15.40:0/3698858642 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb3b4066470 0x7fb3b4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fb3cffff640 1 --2- 172.21.15.40:0/3698858642 >> [v2:172.21.15.40:6800/2328483711,v1:172.21.15.40:6801/2328483711] conn(0x7fb3b406e610 0x7fb3b4070a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fd242dcb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd23c0cddc0 0x7fd23c1500a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fd242dcb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd23c0d0140 0x7fd23c157440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fd242dcb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd23c160510 0x7fd23c157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fd242dcb640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c0cb3a0 con 0x7fd23c160510 2023-10-16T08:38:50.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fd242dcb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c0cc200 con 0x7fd23c0d0140 2023-10-16T08:38:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fd242dcb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd23c0cb5f0 con 0x7fd23c0cddc0 2023-10-16T08:38:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fd241368640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd23c160510 0x7fd23c157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fd241368640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd23c160510 0x7fd23c157980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58218/0 (socket says 172.21.15.40:58218) 2023-10-16T08:38:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fd241368640 1 -- 172.21.15.40:0/2785357715 learned_addr learned my addr 172.21.15.40:0/2785357715 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fd23bfff640 1 --2- 172.21.15.40:0/2785357715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd23c0d0140 0x7fd23c157440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fb3cffff640 1 --2- 172.21.15.40:0/3698858642 >> [v2:172.21.15.40:6800/2328483711,v1:172.21.15.40:6801/2328483711] conn(0x7fb3b406e610 0x7fb3b4070a30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fb3ceffd640 1 --2- 172.21.15.40:0/3698858642 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb3b4066470 0x7fb3b4068930 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80ae000 tx=0x7fb3c80a7070 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fb3bffff640 1 -- 172.21.15.40:0/3698858642 <== osd.2 v2:172.21.15.40:6800/2328483711 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fb3b40710e0 con 0x7fb3b406e610 2023-10-16T08:38:50.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f0f644f4640 1 -- 172.21.15.40:0/1904429788 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0f5c0cca40 con 0x7f0f5c0c4430 2023-10-16T08:38:50.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fa37229e640 1 Processor -- start 2023-10-16T08:38:50.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fd240b67640 1 --2- 172.21.15.40:0/2785357715 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd23c0cddc0 0x7fd23c1500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fd241368640 1 -- 172.21.15.40:0/2785357715 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd23c0cddc0 msgr2=0x7fd23c1500a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fd241368640 1 --2- 172.21.15.40:0/2785357715 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd23c0cddc0 0x7fd23c1500a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fd241368640 1 -- 172.21.15.40:0/2785357715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd23c0d0140 msgr2=0x7fd23c157440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fd241368640 1 --2- 172.21.15.40:0/2785357715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd23c0d0140 0x7fd23c157440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fd241368640 1 -- 172.21.15.40:0/2785357715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd230009420 con 0x7fd23c160510 2023-10-16T08:38:50.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fd241368640 1 --2- 172.21.15.40:0/2785357715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd23c160510 0x7fd23c157980 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd23400c7c0 tx=0x7fd23400cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f408d7c6640 1 Processor -- start 2023-10-16T08:38:50.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f0f52ffd640 1 -- 172.21.15.40:0/1904429788 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0f54026c80 con 0x7f0f5c0c4430 2023-10-16T08:38:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f0f52ffd640 1 --2- 172.21.15.40:0/1904429788 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0f40066570 0x7f0f40068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f0f52ffd640 1 -- 172.21.15.40:0/1904429788 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f540983d0 con 0x7f0f5c0c4430 2023-10-16T08:38:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f0f52ffd640 1 --2- 172.21.15.40:0/1904429788 >> [v2:172.21.15.40:6808/1155811160,v1:172.21.15.40:6809/1155811160] conn(0x7f0f4006e710 0x7f0f40070b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f0f52ffd640 1 -- 172.21.15.40:0/1904429788 --> [v2:172.21.15.40:6808/1155811160,v1:172.21.15.40:6809/1155811160] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0f400711e0 con 0x7f0f4006e710 2023-10-16T08:38:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fa37229e640 1 -- start start 2023-10-16T08:38:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fa37229e640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa36c156a20 0x7fa36c159e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fa37229e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fa36c0cb370 0x7fa36c0cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fa37229e640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa36c0ae0d0 con 0x7fa36c0adb80 2023-10-16T08:38:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fa37229e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa36c0cb0a0 con 0x7fa36c0cb370 2023-10-16T08:38:50.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fa37229e640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa36c156e00 con 0x7fa36c156a20 2023-10-16T08:38:50.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f408d7c6640 1 -- start start 2023-10-16T08:38:50.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f408d7c6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40880cfad0 0x7f40880b3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f408d7c6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f408815d780 0x7f40880ac180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f408d7c6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40880a9ec0 0x7f40880aa370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f408d7c6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40880aa8b0 con 0x7f40880a9ec0 2023-10-16T08:38:50.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f408d7c6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408813f760 con 0x7f408815d780 2023-10-16T08:38:50.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f408d7c6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40880ac6c0 con 0x7f40880cfad0 2023-10-16T08:38:50.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f0f52ffd640 1 -- 172.21.15.40:0/1904429788 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0f540658e0 con 0x7f0f5c0c4430 2023-10-16T08:38:50.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f40877fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f408815d780 0x7f40880ac180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f40877fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f408815d780 0x7f40880ac180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.40:53852/0 (socket says 172.21.15.40:53852) 2023-10-16T08:38:50.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f40877fe640 1 -- 172.21.15.40:0/767372449 learned_addr learned my addr 172.21.15.40:0/767372449 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f408cfc5640 1 --2- 172.21.15.40:0/767372449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40880a9ec0 0x7f40880aa370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f4087fff640 1 --2- 172.21.15.40:0/767372449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40880cfad0 0x7f40880b3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f0f62290640 1 --2- 172.21.15.40:0/1904429788 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0f40066570 0x7f0f40068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fd2397fa640 1 -- 172.21.15.40:0/2785357715 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd234003b80 con 0x7fd23c160510 2023-10-16T08:38:50.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fd2397fa640 1 -- 172.21.15.40:0/2785357715 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd234003ce0 con 0x7fd23c160510 2023-10-16T08:38:50.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fd2397fa640 1 -- 172.21.15.40:0/2785357715 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd23400aa50 con 0x7fd23c160510 2023-10-16T08:38:50.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fa36b7fe640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa36c0adb80 0x7fa36c0cc530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.40:47202/0 (socket says 172.21.15.40:47202) 2023-10-16T08:38:50.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7fa36b7fe640 1 -- 172.21.15.40:0/652153134 learned_addr learned my addr 172.21.15.40:0/652153134 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:50.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.568+0000 7f0f62a91640 1 --2- 172.21.15.40:0/1904429788 >> [v2:172.21.15.40:6808/1155811160,v1:172.21.15.40:6809/1155811160] conn(0x7f0f4006e710 0x7f0f40070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7f408cfc5640 1 -- 172.21.15.40:0/767372449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40880cfad0 msgr2=0x7f40880b3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7f408cfc5640 1 --2- 172.21.15.40:0/767372449 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40880cfad0 0x7f40880b3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7f408cfc5640 1 -- 172.21.15.40:0/767372449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f408815d780 msgr2=0x7f40880ac180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7f408cfc5640 1 --2- 172.21.15.40:0/767372449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f408815d780 0x7f40880ac180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7f408cfc5640 1 -- 172.21.15.40:0/767372449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4080009a70 con 0x7f40880a9ec0 2023-10-16T08:38:50.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7fa36bfff640 1 --2- 172.21.15.40:0/652153134 >> v2:172.21.15.86:3300/0 conn(0x7fa36c156a20 0x7fa36c159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7fd242dcb640 1 -- 172.21.15.40:0/2785357715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd23c0cb8d0 con 0x7fd23c160510 2023-10-16T08:38:50.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7fd242dcb640 1 -- 172.21.15.40:0/2785357715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd23c0cbe90 con 0x7fd23c160510 2023-10-16T08:38:50.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7f0f62a91640 1 --2- 172.21.15.40:0/1904429788 >> [v2:172.21.15.40:6808/1155811160,v1:172.21.15.40:6809/1155811160] conn(0x7f0f4006e710 0x7f0f40070b30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7fa37083b640 1 --2- 172.21.15.40:0/652153134 >> v2:172.21.15.40:3300/0 conn(0x7fa36c0cb370 0x7fa36c0cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7f408cfc5640 1 --2- 172.21.15.40:0/767372449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40880a9ec0 0x7f40880aa370 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f407800c390 tx=0x7f407800c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7f0f62290640 1 --2- 172.21.15.40:0/1904429788 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0f40066570 0x7f0f40068a30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0f580ad000 tx=0x7f0f580a7000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7fa36affd640 1 -- 172.21.15.40:0/652153134 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2723339730 0 0) 0x7fa36c0ae0d0 con 0x7fa36c0adb80 2023-10-16T08:38:50.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7fa36affd640 1 -- 172.21.15.40:0/652153134 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa354003530 con 0x7fa36c0adb80 2023-10-16T08:38:50.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7fd242dcb640 1 -- 172.21.15.40:0/2785357715 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fd23c0d3ba0 con 0x7fd23c160510 2023-10-16T08:38:50.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7f0f52ffd640 1 -- 172.21.15.40:0/1904429788 <== osd.3 v2:172.21.15.40:6808/1155811160 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f0f400711e0 con 0x7f0f4006e710 2023-10-16T08:38:50.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7f4084ff9640 1 -- 172.21.15.40:0/767372449 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40780074e0 con 0x7f40880a9ec0 2023-10-16T08:38:50.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7f4084ff9640 1 -- 172.21.15.40:0/767372449 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4078007640 con 0x7f40880a9ec0 2023-10-16T08:38:50.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7f4084ff9640 1 -- 172.21.15.40:0/767372449 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4078004a10 con 0x7f40880a9ec0 2023-10-16T08:38:50.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7f408d7c6640 1 -- 172.21.15.40:0/767372449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40880b2e00 con 0x7f40880a9ec0 2023-10-16T08:38:50.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7f408d7c6640 1 -- 172.21.15.40:0/767372449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40880b33c0 con 0x7f40880a9ec0 2023-10-16T08:38:50.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7fa36bfff640 1 -- 172.21.15.40:0/652153134 >> v2:172.21.15.40:3300/0 conn(0x7fa36c0cb370 msgr2=0x7fa36c0cb7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7fa36bfff640 1 --2- 172.21.15.40:0/652153134 >> v2:172.21.15.40:3300/0 conn(0x7fa36c0cb370 0x7fa36c0cb7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7fa36bfff640 1 -- 172.21.15.40:0/652153134 >> v1:172.21.15.86:6789/0 conn(0x7fa36c0adb80 legacy=0x7fa36c0cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.572+0000 7fa36bfff640 1 -- 172.21.15.40:0/652153134 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36c0ccc40 con 0x7fa36c156a20 2023-10-16T08:38:50.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7fa36bfff640 1 --2- 172.21.15.40:0/652153134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa36c156a20 0x7fa36c159e60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0cb340 tx=0x7fa35c01e6c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8e5f41ede6483661 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7f408d7c6640 1 -- 172.21.15.40:0/767372449 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4068000fd0 con 0x7f40880a9ec0 2023-10-16T08:38:50.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7fa36affd640 1 -- 172.21.15.40:0/652153134 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c0ad600 con 0x7fa36c156a20 2023-10-16T08:38:50.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7fa36affd640 1 -- 172.21.15.40:0/652153134 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c0a9050 con 0x7fa36c156a20 2023-10-16T08:38:50.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7fa37229e640 1 -- 172.21.15.40:0/652153134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa36c156a20 msgr2=0x7fa36c159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7fa37229e640 1 --2- 172.21.15.40:0/652153134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa36c156a20 0x7fa36c159e60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa36c0cb340 tx=0x7fa35c01e6c0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7f4084ff9640 1 -- 172.21.15.40:0/767372449 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4078004b70 con 0x7f40880a9ec0 2023-10-16T08:38:50.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7f4084ff9640 1 --2- 172.21.15.40:0/767372449 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f4074066570 0x7f4074068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7f4084ff9640 1 -- 172.21.15.40:0/767372449 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4078083530 con 0x7f40880a9ec0 2023-10-16T08:38:50.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7f4084ff9640 1 --2- 172.21.15.40:0/767372449 >> [v2:172.21.15.86:6800/2560845614,v1:172.21.15.86:6801/2560845614] conn(0x7f407406e710 0x7f4074070b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7f4084ff9640 1 -- 172.21.15.40:0/767372449 --> [v2:172.21.15.86:6800/2560845614,v1:172.21.15.86:6801/2560845614] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40740711e0 con 0x7f407406e710 2023-10-16T08:38:50.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7fa37229e640 1 -- 172.21.15.40:0/652153134 shutdown_connections 2023-10-16T08:38:50.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7fa37229e640 1 --2- 172.21.15.40:0/652153134 >> v2:172.21.15.40:3300/0 conn(0x7fa36c0cb370 0x7fa36c0cb7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7fa37229e640 1 --2- 172.21.15.40:0/652153134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa36c156a20 0x7fa36c159e60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7fa37229e640 1 -- 172.21.15.40:0/652153134 >> 172.21.15.40:0/652153134 conn(0x7fa36c0b0470 msgr2=0x7fa36c0b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7f4087fff640 1 --2- 172.21.15.40:0/767372449 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f4074066570 0x7f4074068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7f40877fe640 1 --2- 172.21.15.40:0/767372449 >> [v2:172.21.15.86:6800/2560845614,v1:172.21.15.86:6801/2560845614] conn(0x7f407406e710 0x7f4074070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7fa37229e640 1 -- 172.21.15.40:0/652153134 shutdown_connections 2023-10-16T08:38:50.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7fd2397fa640 1 -- 172.21.15.40:0/2785357715 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd23400abb0 con 0x7fd23c160510 2023-10-16T08:38:50.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7fd2397fa640 1 --2- 172.21.15.40:0/2785357715 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd220066570 0x7fd220068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7fd2397fa640 1 -- 172.21.15.40:0/2785357715 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd234084660 con 0x7fd23c160510 2023-10-16T08:38:50.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7fd2397fa640 1 --2- 172.21.15.40:0/2785357715 >> [v2:172.21.15.86:6824/751032064,v1:172.21.15.86:6825/751032064] conn(0x7fd22006e790 0x7fd220070bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7fd2397fa640 1 -- 172.21.15.40:0/2785357715 --> [v2:172.21.15.86:6824/751032064,v1:172.21.15.86:6825/751032064] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd220071260 con 0x7fd22006e790 2023-10-16T08:38:50.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7fd2397fa640 1 -- 172.21.15.40:0/2785357715 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fd234084a10 con 0x7fd23c160510 2023-10-16T08:38:50.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7fd240b67640 1 --2- 172.21.15.40:0/2785357715 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd220066570 0x7fd220068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7fd23bfff640 1 --2- 172.21.15.40:0/2785357715 >> [v2:172.21.15.86:6824/751032064,v1:172.21.15.86:6825/751032064] conn(0x7fd22006e790 0x7fd220070bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7f4084ff9640 1 -- 172.21.15.40:0/767372449 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4078050a40 con 0x7f40880a9ec0 2023-10-16T08:38:50.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7f510e4ec640 1 Processor -- start 2023-10-16T08:38:50.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7f40877fe640 1 --2- 172.21.15.40:0/767372449 >> [v2:172.21.15.86:6800/2560845614,v1:172.21.15.86:6801/2560845614] conn(0x7f407406e710 0x7f4074070b30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7fd240b67640 1 --2- 172.21.15.40:0/2785357715 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd220066570 0x7fd220068a30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd228009ab0 tx=0x7fd228009240 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7fd23bfff640 1 --2- 172.21.15.40:0/2785357715 >> [v2:172.21.15.86:6824/751032064,v1:172.21.15.86:6825/751032064] conn(0x7fd22006e790 0x7fd220070bb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7f510e4ec640 1 -- start start 2023-10-16T08:38:50.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7f510e4ec640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5108159be0 0x7f51080cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7f510e4ec640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f51080ce370 0x7f51080ca2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7f510e4ec640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5108159fc0 con 0x7f51080ca800 2023-10-16T08:38:50.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7f510e4ec640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f510806c450 con 0x7f51080ce370 2023-10-16T08:38:50.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7f510e4ec640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f510806c730 con 0x7f5108159be0 2023-10-16T08:38:50.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7f51077fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f51080ce370 0x7f51080ca2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7f51077fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f51080ce370 0x7f51080ca2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58234/0 (socket says 172.21.15.40:58234) 2023-10-16T08:38:50.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7f51077fe640 1 -- 172.21.15.40:0/1450184057 learned_addr learned my addr 172.21.15.40:0/1450184057 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.576+0000 7fa37229e640 1 -- 172.21.15.40:0/652153134 wait complete. 2023-10-16T08:38:50.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7fa37229e640 1 Processor -- start 2023-10-16T08:38:50.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7f5107fff640 1 --2- 172.21.15.40:0/1450184057 >> v2:172.21.15.86:3300/0 conn(0x7f5108159be0 0x7f51080cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7f51077fe640 1 -- 172.21.15.40:0/1450184057 >> v2:172.21.15.86:3300/0 conn(0x7f5108159be0 msgr2=0x7f51080cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7f51077fe640 1 --2- 172.21.15.40:0/1450184057 >> v2:172.21.15.86:3300/0 conn(0x7f5108159be0 0x7f51080cde30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7f51077fe640 1 -- 172.21.15.40:0/1450184057 >> v1:172.21.15.40:6789/0 conn(0x7f51080ca800 legacy=0x7f51080b1360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7f51077fe640 1 -- 172.21.15.40:0/1450184057 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f510806ca10 con 0x7f51080ce370 2023-10-16T08:38:50.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7f4084ff9640 1 -- 172.21.15.40:0/767372449 <== osd.7 v2:172.21.15.86:6800/2560845614 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f40740711e0 con 0x7f407406e710 2023-10-16T08:38:50.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7fa37229e640 1 -- start start 2023-10-16T08:38:50.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7fa37229e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa36c0adb80 0x7fa36c2036b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7fa37229e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa36c0cb370 0x7fa36c0c6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7f4087fff640 1 --2- 172.21.15.40:0/767372449 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f4074066570 0x7f4074068a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f40800af000 tx=0x7f4080004ac0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7fa36bfff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa36c0adb80 0x7fa36c2036b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7fa36bfff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa36c0adb80 0x7fa36c2036b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.40:54512/0 (socket says 172.21.15.40:54512) 2023-10-16T08:38:50.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7fa36b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa36c0cb370 0x7fa36c0c6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7fa36b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa36c0cb370 0x7fa36c0c6440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58238/0 (socket says 172.21.15.40:58238) 2023-10-16T08:38:50.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7fa36bfff640 1 -- 172.21.15.40:0/4095320081 learned_addr learned my addr 172.21.15.40:0/4095320081 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7fa37229e640 1 --2- 172.21.15.40:0/4095320081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa36c156a20 0x7fa36c0c6980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7fa37229e640 1 -- 172.21.15.40:0/4095320081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c152410 con 0x7fa36c0cb370 2023-10-16T08:38:50.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7fa37229e640 1 -- 172.21.15.40:0/4095320081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c151870 con 0x7fa36c156a20 2023-10-16T08:38:50.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7fa37229e640 1 -- 172.21.15.40:0/4095320081 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36c151ae0 con 0x7fa36c0adb80 2023-10-16T08:38:50.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.580+0000 7fa37083b640 1 --2- 172.21.15.40:0/4095320081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa36c156a20 0x7fa36c0c6980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7fa36b7fe640 1 -- 172.21.15.40:0/4095320081 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa36c0adb80 msgr2=0x7fa36c2036b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7fa36b7fe640 1 --2- 172.21.15.40:0/4095320081 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa36c0adb80 0x7fa36c2036b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7fa36b7fe640 1 -- 172.21.15.40:0/4095320081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa36c156a20 msgr2=0x7fa36c0c6980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7fa36b7fe640 1 --2- 172.21.15.40:0/4095320081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa36c156a20 0x7fa36c0c6980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7fa36b7fe640 1 -- 172.21.15.40:0/4095320081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa35c009420 con 0x7fa36c0cb370 2023-10-16T08:38:50.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7f51077fe640 1 --2- 172.21.15.40:0/1450184057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51080ce370 0x7f51080ca2c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f510806aee0 tx=0x7f50f801ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be90f65743f8cc75 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7fa36b7fe640 1 --2- 172.21.15.40:0/4095320081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa36c0cb370 0x7fa36c0c6440 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa36000d5b0 tx=0x7fa360001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7f5106ffd640 1 -- 172.21.15.40:0/1450184057 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50f80a7ba0 con 0x7f51080ce370 2023-10-16T08:38:50.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7fa368ff9640 1 -- 172.21.15.40:0/4095320081 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36000f850 con 0x7fa36c0cb370 2023-10-16T08:38:50.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7f5106ffd640 1 -- 172.21.15.40:0/1450184057 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50f80a7d00 con 0x7f51080ce370 2023-10-16T08:38:50.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7f510e4ec640 1 -- 172.21.15.40:0/1450184057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51080ce370 msgr2=0x7f51080ca2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7f510e4ec640 1 --2- 172.21.15.40:0/1450184057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51080ce370 0x7f51080ca2c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f510806aee0 tx=0x7f50f801ead0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7f510e4ec640 1 -- 172.21.15.40:0/1450184057 shutdown_connections 2023-10-16T08:38:50.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7f510e4ec640 1 --2- 172.21.15.40:0/1450184057 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f51080ce370 0x7f51080ca2c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7f510e4ec640 1 --2- 172.21.15.40:0/1450184057 >> v2:172.21.15.86:3300/0 conn(0x7f5108159be0 0x7f51080cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7f510e4ec640 1 -- 172.21.15.40:0/1450184057 >> 172.21.15.40:0/1450184057 conn(0x7f51080cc360 msgr2=0x7f51080cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7f510e4ec640 1 -- 172.21.15.40:0/1450184057 shutdown_connections 2023-10-16T08:38:50.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7fa37229e640 1 -- 172.21.15.40:0/4095320081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36c151dc0 con 0x7fa36c0cb370 2023-10-16T08:38:50.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7fa37229e640 1 -- 172.21.15.40:0/4095320081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36c203bf0 con 0x7fa36c0cb370 2023-10-16T08:38:50.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7fa37229e640 1 -- 172.21.15.40:0/4095320081 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa36c0cc6f0 con 0x7fa36c0cb370 2023-10-16T08:38:50.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7fa368ff9640 1 -- 172.21.15.40:0/4095320081 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa36000f9b0 con 0x7fa36c0cb370 2023-10-16T08:38:50.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7fa368ff9640 1 -- 172.21.15.40:0/4095320081 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa36000fd60 con 0x7fa36c0cb370 2023-10-16T08:38:50.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7fa368ff9640 1 -- 172.21.15.40:0/4095320081 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa360010420 con 0x7fa36c0cb370 2023-10-16T08:38:50.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f1d0054d640 1 Processor -- start 2023-10-16T08:38:50.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.584+0000 7f510e4ec640 1 -- 172.21.15.40:0/1450184057 wait complete. 2023-10-16T08:38:50.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f1d0054d640 1 -- start start 2023-10-16T08:38:50.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7fd2397fa640 1 -- 172.21.15.40:0/2785357715 <== osd.5 v2:172.21.15.86:6824/751032064 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fd220071260 con 0x7fd22006e790 2023-10-16T08:38:50.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f510e4ec640 1 Processor -- start 2023-10-16T08:38:50.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f1d0054d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1cf80caec0 0x7f1cf80cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f1d0054d640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1cf80c9960 0x7f1cf80ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f510e4ec640 1 -- start start 2023-10-16T08:38:50.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f510e4ec640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f51080ca800 0x7f51080c3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f510e4ec640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5108159be0 0x7f5108157d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.657 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f1d0054d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cf80cd0e0 0x7f1cf815a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f1cfe2e9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1cf80caec0 0x7f1cf80cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f1d0054d640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf80cb7e0 con 0x7f1cf80caec0 2023-10-16T08:38:50.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f1cfe2e9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1cf80caec0 0x7f1cf80cb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58246/0 (socket says 172.21.15.40:58246) 2023-10-16T08:38:50.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f1cfe2e9640 1 -- 172.21.15.40:0/1515370012 learned_addr learned my addr 172.21.15.40:0/1515370012 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f1d0054d640 1 -- 172.21.15.40:0/1515370012 --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cf8158f30 con 0x7f1cf80c9960 2023-10-16T08:38:50.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f1d0054d640 1 -- 172.21.15.40:0/1515370012 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf80c8ba0 con 0x7f1cf80cd0e0 2023-10-16T08:38:50.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f5107fff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f51080ca800 0x7f51080c3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f1cfdae8640 1 --2- 172.21.15.40:0/1515370012 >> v2:172.21.15.86:3300/0 conn(0x7f1cf80c9960 0x7f1cf80ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f5107fff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f51080ca800 0x7f51080c3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.40:54528/0 (socket says 172.21.15.40:54528) 2023-10-16T08:38:50.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f5107fff640 1 -- 172.21.15.40:0/2357837184 learned_addr learned my addr 172.21.15.40:0/2357837184 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f1cfeaea640 1 --2- 172.21.15.40:0/1515370012 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cf80cd0e0 0x7f1cf815a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f510e4ec640 1 --2- 172.21.15.40:0/2357837184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f510815cbd0 0x7f51080c8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f51077fe640 1 --2- 172.21.15.40:0/2357837184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5108159be0 0x7f5108157d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f510ca89640 1 --2- 172.21.15.40:0/2357837184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f510815cbd0 0x7f51080c8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f510e4ec640 1 -- 172.21.15.40:0/2357837184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510815c680 con 0x7f510815cbd0 2023-10-16T08:38:50.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f510e4ec640 1 -- 172.21.15.40:0/2357837184 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f510815d060 con 0x7f5108159be0 2023-10-16T08:38:50.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7f510e4ec640 1 -- 172.21.15.40:0/2357837184 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51080c41b0 con 0x7f51080ca800 2023-10-16T08:38:50.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7f510ca89640 1 -- 172.21.15.40:0/2357837184 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f51080ca800 msgr2=0x7f51080c3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7f510ca89640 1 --2- 172.21.15.40:0/2357837184 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f51080ca800 0x7f51080c3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7f510ca89640 1 -- 172.21.15.40:0/2357837184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5108159be0 msgr2=0x7f5108157d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7f510ca89640 1 --2- 172.21.15.40:0/2357837184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5108159be0 0x7f5108157d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7f510ca89640 1 -- 172.21.15.40:0/2357837184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50f800b770 con 0x7f510815cbd0 2023-10-16T08:38:50.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7f51077fe640 1 --2- 172.21.15.40:0/2357837184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5108159be0 0x7f5108157d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:38:50.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7f510ca89640 1 --2- 172.21.15.40:0/2357837184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f510815cbd0 0x7f51080c8cb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f510000c780 tx=0x7f510000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f1cfeaea640 1 -- 172.21.15.40:0/1515370012 >> v2:172.21.15.86:3300/0 conn(0x7f1cf80c9960 msgr2=0x7f1cf80ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f1cfeaea640 1 --2- 172.21.15.40:0/1515370012 >> v2:172.21.15.86:3300/0 conn(0x7f1cf80c9960 0x7f1cf80ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f1cfeaea640 1 -- 172.21.15.40:0/1515370012 >> v2:172.21.15.40:3300/0 conn(0x7f1cf80caec0 msgr2=0x7f1cf80cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f1cfeaea640 1 --2- 172.21.15.40:0/1515370012 >> v2:172.21.15.40:3300/0 conn(0x7f1cf80caec0 0x7f1cf80cb2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.588+0000 7f1cfeaea640 1 -- 172.21.15.40:0/1515370012 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf80c8e80 con 0x7f1cf80cd0e0 2023-10-16T08:38:50.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7f1cfe2e9640 1 --2- 172.21.15.40:0/1515370012 >> v2:172.21.15.40:3300/0 conn(0x7f1cf80caec0 0x7f1cf80cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:38:50.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7f1cfeaea640 1 --2- 172.21.15.40:0/1515370012 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cf80cd0e0 0x7f1cf815a260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80ca8d0 tx=0x7f1cf001ccc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b484d10ca5609978 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7f5104ff9640 1 -- 172.21.15.40:0/2357837184 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510000aa50 con 0x7f510815cbd0 2023-10-16T08:38:50.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7f5104ff9640 1 -- 172.21.15.40:0/2357837184 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f510000abb0 con 0x7f510815cbd0 2023-10-16T08:38:50.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7f5104ff9640 1 -- 172.21.15.40:0/2357837184 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f510000ce30 con 0x7f510815cbd0 2023-10-16T08:38:50.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7f510e4ec640 1 -- 172.21.15.40:0/2357837184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f510815bfc0 con 0x7f510815cbd0 2023-10-16T08:38:50.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7f510e4ec640 1 -- 172.21.15.40:0/2357837184 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51080c91f0 con 0x7f510815cbd0 2023-10-16T08:38:50.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7fa368ff9640 1 --2- 172.21.15.40:0/4095320081 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa354066a70 0x7fa354068f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7fa368ff9640 1 -- 172.21.15.40:0/4095320081 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa360009040 con 0x7fa36c0cb370 2023-10-16T08:38:50.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7fa368ff9640 1 --2- 172.21.15.40:0/4095320081 >> [v2:172.21.15.86:6808/150800249,v1:172.21.15.86:6809/150800249] conn(0x7fa35406eb20 0x7fa354070f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7fa368ff9640 1 -- 172.21.15.40:0/4095320081 --> [v2:172.21.15.86:6808/150800249,v1:172.21.15.86:6809/150800249] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa3540715f0 con 0x7fa35406eb20 2023-10-16T08:38:50.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7fa368ff9640 1 -- 172.21.15.40:0/4095320081 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa360086600 con 0x7fa36c0cb370 2023-10-16T08:38:50.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7fa36bfff640 1 --2- 172.21.15.40:0/4095320081 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa354066a70 0x7fa354068f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7fa37083b640 1 --2- 172.21.15.40:0/4095320081 >> [v2:172.21.15.86:6808/150800249,v1:172.21.15.86:6809/150800249] conn(0x7fa35406eb20 0x7fa354070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7fa36bfff640 1 --2- 172.21.15.40:0/4095320081 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa354066a70 0x7fa354068f30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa35c01e700 tx=0x7fa35c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7fa37083b640 1 --2- 172.21.15.40:0/4095320081 >> [v2:172.21.15.86:6808/150800249,v1:172.21.15.86:6809/150800249] conn(0x7fa35406eb20 0x7fa354070f40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.592+0000 7fa368ff9640 1 -- 172.21.15.40:0/4095320081 <== osd.4 v2:172.21.15.86:6808/150800249 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fa3540715f0 con 0x7fa35406eb20 2023-10-16T08:38:50.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7fb3d6199640 1 -- 172.21.15.40:0/3698858642 --> [v2:172.21.15.40:6800/2328483711,v1:172.21.15.40:6801/2328483711] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fb3d00c3510 con 0x7fb3b406e610 2023-10-16T08:38:50.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7fb3bffff640 1 -- 172.21.15.40:0/3698858642 <== osd.2 v2:172.21.15.40:6800/2328483711 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fb3d00c3510 con 0x7fb3b406e610 2023-10-16T08:38:50.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7f5104ff9640 1 -- 172.21.15.40:0/2357837184 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5100012420 con 0x7f510815cbd0 2023-10-16T08:38:50.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7f5104ff9640 1 --2- 172.21.15.40:0/2357837184 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f50f4066570 0x7f50f4068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7f5104ff9640 1 -- 172.21.15.40:0/2357837184 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5100084330 con 0x7f510815cbd0 2023-10-16T08:38:50.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7fb3d6199640 1 -- 172.21.15.40:0/3698858642 >> [v2:172.21.15.40:6800/2328483711,v1:172.21.15.40:6801/2328483711] conn(0x7fb3b406e610 msgr2=0x7fb3b4070a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7fb3d6199640 1 --2- 172.21.15.40:0/3698858642 >> [v2:172.21.15.40:6800/2328483711,v1:172.21.15.40:6801/2328483711] conn(0x7fb3b406e610 0x7fb3b4070a30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7fb3d6199640 1 -- 172.21.15.40:0/3698858642 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb3b4066470 msgr2=0x7fb3b4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7fb3d6199640 1 --2- 172.21.15.40:0/3698858642 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb3b4066470 0x7fb3b4068930 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fb3c80ae000 tx=0x7fb3c80a7070 comp rx=0 tx=0).stop 2023-10-16T08:38:50.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7fb3d6199640 1 -- 172.21.15.40:0/3698858642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d00cde30 msgr2=0x7fb3d00b2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7fb3d6199640 1 --2- 172.21.15.40:0/3698858642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d00cde30 0x7fb3d00b2140 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fb3c000d5f0 tx=0x7fb3c000dab0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7fb3cffff640 1 -- 172.21.15.40:0/3698858642 reap_dead start 2023-10-16T08:38:50.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7fb3d6199640 1 -- 172.21.15.40:0/3698858642 shutdown_connections 2023-10-16T08:38:50.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7fb3d6199640 1 -- 172.21.15.40:0/3698858642 >> 172.21.15.40:0/3698858642 conn(0x7fb3d00cb090 msgr2=0x7fb3d00cb470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7fb3d6199640 1 -- 172.21.15.40:0/3698858642 shutdown_connections 2023-10-16T08:38:50.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7f510e4ec640 1 --2- 172.21.15.40:0/2357837184 >> [v2:172.21.15.40:6817/2222522452,v1:172.21.15.40:6819/2222522452] conn(0x7f51080b2460 0x7f51080b2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7fb3d6199640 1 -- 172.21.15.40:0/3698858642 wait complete. 2023-10-16T08:38:50.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7f1cfd2e7640 1 -- 172.21.15.40:0/1515370012 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf00a5d50 con 0x7f1cf80cd0e0 2023-10-16T08:38:50.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7f5107fff640 1 --2- 172.21.15.40:0/2357837184 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f50f4066570 0x7f50f4068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7f51077fe640 1 --2- 172.21.15.40:0/2357837184 >> [v2:172.21.15.40:6817/2222522452,v1:172.21.15.40:6819/2222522452] conn(0x7f51080b2460 0x7f51080b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7f510e4ec640 1 -- 172.21.15.40:0/2357837184 --> [v2:172.21.15.40:6817/2222522452,v1:172.21.15.40:6819/2222522452] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f510804e9c0 con 0x7f51080b2460 2023-10-16T08:38:50.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.596+0000 7f1cfd2e7640 1 -- 172.21.15.40:0/1515370012 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf00ad440 con 0x7f1cf80cd0e0 2023-10-16T08:38:50.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.600+0000 7f5107fff640 1 --2- 172.21.15.40:0/2357837184 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f50f4066570 0x7f50f4068a30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00a9d0 tx=0x7f50fc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.600+0000 7f51077fe640 1 --2- 172.21.15.40:0/2357837184 >> [v2:172.21.15.40:6817/2222522452,v1:172.21.15.40:6819/2222522452] conn(0x7f51080b2460 0x7f51080b2840 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.600+0000 7f1d0054d640 1 -- 172.21.15.40:0/1515370012 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cf80cd0e0 msgr2=0x7f1cf815a260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.600+0000 7f1d0054d640 1 --2- 172.21.15.40:0/1515370012 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cf80cd0e0 0x7f1cf815a260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1cf80ca8d0 tx=0x7f1cf001ccc0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.600+0000 7f1d0054d640 1 -- 172.21.15.40:0/1515370012 shutdown_connections 2023-10-16T08:38:50.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.600+0000 7f1d0054d640 1 --2- 172.21.15.40:0/1515370012 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cf80cd0e0 0x7f1cf815a260 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.600+0000 7f1d0054d640 1 --2- 172.21.15.40:0/1515370012 >> v2:172.21.15.86:3300/0 conn(0x7f1cf80c9960 0x7f1cf80ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.600+0000 7f1d0054d640 1 --2- 172.21.15.40:0/1515370012 >> v2:172.21.15.40:3300/0 conn(0x7f1cf80caec0 0x7f1cf80cb2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.600+0000 7f1d0054d640 1 -- 172.21.15.40:0/1515370012 >> 172.21.15.40:0/1515370012 conn(0x7f1cf80affc0 msgr2=0x7f1cf80b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.600+0000 7f1d0054d640 1 -- 172.21.15.40:0/1515370012 shutdown_connections 2023-10-16T08:38:50.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.600+0000 7f1d0054d640 1 -- 172.21.15.40:0/1515370012 wait complete. 2023-10-16T08:38:50.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.600+0000 7f1d0054d640 1 Processor -- start 2023-10-16T08:38:50.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.600+0000 7f1d0054d640 1 -- start start 2023-10-16T08:38:50.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1d0054d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cf80c9960 0x7f1cf8155610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1d0054d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cf80caec0 0x7f1cf80c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1d0054d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cf80cd0e0 0x7f1cf8070c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1d0054d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf806f960 con 0x7f1cf80caec0 2023-10-16T08:38:50.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1d0054d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf80707c0 con 0x7f1cf80cd0e0 2023-10-16T08:38:50.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1d0054d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf8155b50 con 0x7f1cf80c9960 2023-10-16T08:38:50.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1cfe2e9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cf80c9960 0x7f1cf8155610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1cfdae8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cf80caec0 0x7f1cf80c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1cfe2e9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cf80c9960 0x7f1cf8155610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.40:54556/0 (socket says 172.21.15.40:54556) 2023-10-16T08:38:50.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1cfe2e9640 1 -- 172.21.15.40:0/756309401 learned_addr learned my addr 172.21.15.40:0/756309401 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1cfdae8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cf80caec0 0x7f1cf80c46b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58266/0 (socket says 172.21.15.40:58266) 2023-10-16T08:38:50.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1cfeaea640 1 --2- 172.21.15.40:0/756309401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cf80cd0e0 0x7f1cf8070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1cfdae8640 1 -- 172.21.15.40:0/756309401 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cf80c9960 msgr2=0x7f1cf8155610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1cfdae8640 1 --2- 172.21.15.40:0/756309401 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1cf80c9960 0x7f1cf8155610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1cfdae8640 1 -- 172.21.15.40:0/756309401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cf80cd0e0 msgr2=0x7f1cf8070c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1cfdae8640 1 --2- 172.21.15.40:0/756309401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cf80cd0e0 0x7f1cf8070c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.682 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1cfdae8640 1 -- 172.21.15.40:0/756309401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ce8009480 con 0x7f1cf80caec0 2023-10-16T08:38:50.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1cfeaea640 1 --2- 172.21.15.40:0/756309401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1cf80cd0e0 0x7f1cf8070c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:38:50.683 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1cfdae8640 1 --2- 172.21.15.40:0/756309401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cf80caec0 0x7f1cf80c46b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400e870 tx=0x7f1cf400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f408d7c6640 1 -- 172.21.15.40:0/767372449 --> [v2:172.21.15.86:6800/2560845614,v1:172.21.15.86:6801/2560845614] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4068002cc0 con 0x7f407406e710 2023-10-16T08:38:50.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f4084ff9640 1 -- 172.21.15.40:0/767372449 <== osd.7 v2:172.21.15.86:6800/2560845614 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4068002cc0 con 0x7f407406e710 2023-10-16T08:38:50.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f408d7c6640 1 -- 172.21.15.40:0/767372449 >> [v2:172.21.15.86:6800/2560845614,v1:172.21.15.86:6801/2560845614] conn(0x7f407406e710 msgr2=0x7f4074070b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f408d7c6640 1 --2- 172.21.15.40:0/767372449 >> [v2:172.21.15.86:6800/2560845614,v1:172.21.15.86:6801/2560845614] conn(0x7f407406e710 0x7f4074070b30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f408d7c6640 1 -- 172.21.15.40:0/767372449 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f4074066570 msgr2=0x7f4074068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f408d7c6640 1 --2- 172.21.15.40:0/767372449 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f4074066570 0x7f4074068a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f40800af000 tx=0x7f4080004ac0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f408d7c6640 1 -- 172.21.15.40:0/767372449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40880a9ec0 msgr2=0x7f40880aa370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f408d7c6640 1 --2- 172.21.15.40:0/767372449 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40880a9ec0 0x7f40880aa370 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f407800c390 tx=0x7f407800c850 comp rx=0 tx=0).stop 2023-10-16T08:38:50.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f408cfc5640 1 -- 172.21.15.40:0/767372449 reap_dead start 2023-10-16T08:38:50.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f408d7c6640 1 -- 172.21.15.40:0/767372449 shutdown_connections 2023-10-16T08:38:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f408d7c6640 1 -- 172.21.15.40:0/767372449 >> 172.21.15.40:0/767372449 conn(0x7f408806a290 msgr2=0x7f4088142720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f408d7c6640 1 -- 172.21.15.40:0/767372449 shutdown_connections 2023-10-16T08:38:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f408d7c6640 1 -- 172.21.15.40:0/767372449 wait complete. 2023-10-16T08:38:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.600+0000 7f5104ff9640 1 -- 172.21.15.40:0/2357837184 <== osd.1 v2:172.21.15.40:6817/2222522452 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f510804e9c0 con 0x7f51080b2460 2023-10-16T08:38:50.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f0f644f4640 1 -- 172.21.15.40:0/1904429788 --> [v2:172.21.15.40:6808/1155811160,v1:172.21.15.40:6809/1155811160] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f0f5c0cc760 con 0x7f0f4006e710 2023-10-16T08:38:50.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1ceeffd640 1 -- 172.21.15.40:0/756309401 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf4009e00 con 0x7f1cf80caec0 2023-10-16T08:38:50.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.604+0000 7f1ceeffd640 1 -- 172.21.15.40:0/756309401 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cf400c940 con 0x7f1cf80caec0 2023-10-16T08:38:50.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f1ceeffd640 1 -- 172.21.15.40:0/756309401 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cf400ccb0 con 0x7f1cf80caec0 2023-10-16T08:38:50.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f0f52ffd640 1 -- 172.21.15.40:0/1904429788 <== osd.3 v2:172.21.15.40:6808/1155811160 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f0f4c0059a0 con 0x7f0f4006e710 2023-10-16T08:38:50.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f1d0054d640 1 -- 172.21.15.40:0/756309401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf0009a30 con 0x7f1cf80caec0 2023-10-16T08:38:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f1d0054d640 1 -- 172.21.15.40:0/756309401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf806fe90 con 0x7f1cf80caec0 2023-10-16T08:38:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f0f644f4640 1 -- 172.21.15.40:0/1904429788 >> [v2:172.21.15.40:6808/1155811160,v1:172.21.15.40:6809/1155811160] conn(0x7f0f4006e710 msgr2=0x7f0f40070b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f0f644f4640 1 --2- 172.21.15.40:0/1904429788 >> [v2:172.21.15.40:6808/1155811160,v1:172.21.15.40:6809/1155811160] conn(0x7f0f4006e710 0x7f0f40070b30 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7fd242dcb640 1 -- 172.21.15.40:0/2785357715 --> [v2:172.21.15.86:6824/751032064,v1:172.21.15.86:6825/751032064] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd23c04e9c0 con 0x7fd22006e790 2023-10-16T08:38:50.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f0f644f4640 1 -- 172.21.15.40:0/1904429788 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0f40066570 msgr2=0x7f0f40068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f0f644f4640 1 --2- 172.21.15.40:0/1904429788 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0f40066570 0x7f0f40068a30 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f0f580ad000 tx=0x7f0f580a7000 comp rx=0 tx=0).stop 2023-10-16T08:38:50.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f0f644f4640 1 -- 172.21.15.40:0/1904429788 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f5c0c4430 msgr2=0x7f0f5c0c78b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f0f644f4640 1 --2- 172.21.15.40:0/1904429788 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f5c0c4430 0x7f0f5c0c78b0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0f540079a0 tx=0x7f0f540225c0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f0f62a91640 1 -- 172.21.15.40:0/1904429788 reap_dead start 2023-10-16T08:38:50.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f0f644f4640 1 -- 172.21.15.40:0/1904429788 shutdown_connections 2023-10-16T08:38:50.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f0f644f4640 1 -- 172.21.15.40:0/1904429788 >> 172.21.15.40:0/1904429788 conn(0x7f0f5c0cb090 msgr2=0x7f0f5c0ae070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f0f644f4640 1 -- 172.21.15.40:0/1904429788 shutdown_connections 2023-10-16T08:38:50.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f0f644f4640 1 -- 172.21.15.40:0/1904429788 wait complete. 2023-10-16T08:38:50.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f1d0054d640 1 -- 172.21.15.40:0/756309401 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1cf80cc0c0 con 0x7f1cf80caec0 2023-10-16T08:38:50.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7fd2397fa640 1 -- 172.21.15.40:0/2785357715 <== osd.5 v2:172.21.15.86:6824/751032064 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd23c04e9c0 con 0x7fd22006e790 2023-10-16T08:38:50.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f1ceeffd640 1 -- 172.21.15.40:0/756309401 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1cf4013470 con 0x7f1cf80caec0 2023-10-16T08:38:50.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f1ceeffd640 1 --2- 172.21.15.40:0/756309401 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1cdc066570 0x7f1cdc068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f1ceeffd640 1 -- 172.21.15.40:0/756309401 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cf4084160 con 0x7f1cf80caec0 2023-10-16T08:38:50.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f1ceeffd640 1 --2- 172.21.15.40:0/756309401 >> [v2:172.21.15.40:6816/3244406559,v1:172.21.15.40:6818/3244406559] conn(0x7f1cdc06e5a0 0x7f1cdc0709c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7f1ceeffd640 1 -- 172.21.15.40:0/756309401 --> [v2:172.21.15.40:6816/3244406559,v1:172.21.15.40:6818/3244406559] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1cdc071070 con 0x7f1cdc06e5a0 2023-10-16T08:38:50.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.612+0000 7f1cfe2e9640 1 --2- 172.21.15.40:0/756309401 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1cdc066570 0x7f1cdc068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.612+0000 7f1ceeffd640 1 -- 172.21.15.40:0/756309401 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=20) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1cf40515e0 con 0x7f1cf80caec0 2023-10-16T08:38:50.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.612+0000 7f1cfe2e9640 1 --2- 172.21.15.40:0/756309401 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1cdc066570 0x7f1cdc068a30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8006db0 tx=0x7f1ce8009210 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7fd242dcb640 1 -- 172.21.15.40:0/2785357715 >> [v2:172.21.15.86:6824/751032064,v1:172.21.15.86:6825/751032064] conn(0x7fd22006e790 msgr2=0x7fd220070bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7fd242dcb640 1 --2- 172.21.15.40:0/2785357715 >> [v2:172.21.15.86:6824/751032064,v1:172.21.15.86:6825/751032064] conn(0x7fd22006e790 0x7fd220070bb0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7fd242dcb640 1 -- 172.21.15.40:0/2785357715 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd220066570 msgr2=0x7fd220068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7fd242dcb640 1 --2- 172.21.15.40:0/2785357715 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd220066570 0x7fd220068a30 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd228009ab0 tx=0x7fd228009240 comp rx=0 tx=0).stop 2023-10-16T08:38:50.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7fd242dcb640 1 -- 172.21.15.40:0/2785357715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd23c160510 msgr2=0x7fd23c157980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7fd242dcb640 1 --2- 172.21.15.40:0/2785357715 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd23c160510 0x7fd23c157980 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fd23400c7c0 tx=0x7fd23400cc80 comp rx=0 tx=0).stop 2023-10-16T08:38:50.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7fd241368640 1 -- 172.21.15.40:0/2785357715 reap_dead start 2023-10-16T08:38:50.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7fd242dcb640 1 -- 172.21.15.40:0/2785357715 shutdown_connections 2023-10-16T08:38:50.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7fd242dcb640 1 -- 172.21.15.40:0/2785357715 >> 172.21.15.40:0/2785357715 conn(0x7fd23c06a060 msgr2=0x7fd23c0c6220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7fd242dcb640 1 -- 172.21.15.40:0/2785357715 shutdown_connections 2023-10-16T08:38:50.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.608+0000 7fd242dcb640 1 -- 172.21.15.40:0/2785357715 wait complete. 2023-10-16T08:38:50.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.616+0000 7fa37229e640 1 -- 172.21.15.40:0/4095320081 --> [v2:172.21.15.86:6808/150800249,v1:172.21.15.86:6809/150800249] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa36c0cbd20 con 0x7fa35406eb20 2023-10-16T08:38:50.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.612+0000 7fe841b1c640 1 Processor -- start 2023-10-16T08:38:50.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.612+0000 7fe841b1c640 1 -- start start 2023-10-16T08:38:50.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.612+0000 7fe841b1c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe83c0ca6b0 0x7fe83c0cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.612+0000 7fe841b1c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe83c0caa90 0x7fe83c08a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.699 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.612+0000 7fe841b1c640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe83c0cd430 con 0x7fe83c071170 2023-10-16T08:38:50.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.612+0000 7fe841b1c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe83c06cb00 con 0x7fe83c0caa90 2023-10-16T08:38:50.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.612+0000 7fe841b1c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c0cd960 con 0x7fe83c0ca6b0 2023-10-16T08:38:50.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.612+0000 7fe84131b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe83c0caa90 0x7fe83c08a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.612+0000 7fe84131b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe83c0caa90 0x7fe83c08a7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58268/0 (socket says 172.21.15.40:58268) 2023-10-16T08:38:50.700 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.612+0000 7fe84131b640 1 -- 172.21.15.40:0/728071304 learned_addr learned my addr 172.21.15.40:0/728071304 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.612+0000 7fe840b1a640 1 --2- 172.21.15.40:0/728071304 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe83c0ca6b0 0x7fe83c0cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.612+0000 7fe83b7fe640 1 -- 172.21.15.40:0/728071304 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2420288109 0 0) 0x7fe83c0cd430 con 0x7fe83c071170 2023-10-16T08:38:50.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.616+0000 7fe83b7fe640 1 -- 172.21.15.40:0/728071304 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe82c003430 con 0x7fe83c071170 2023-10-16T08:38:50.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.616+0000 7fe84131b640 1 -- 172.21.15.40:0/728071304 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe83c0ca6b0 msgr2=0x7fe83c0cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.616+0000 7fe84131b640 1 --2- 172.21.15.40:0/728071304 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe83c0ca6b0 0x7fe83c0cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.616+0000 7fe84131b640 1 -- 172.21.15.40:0/728071304 >> v1:172.21.15.86:6789/0 conn(0x7fe83c071170 legacy=0x7fe83c08a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.616+0000 7fe84131b640 1 -- 172.21.15.40:0/728071304 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe83c0cd680 con 0x7fe83c0caa90 2023-10-16T08:38:50.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.616+0000 7fa368ff9640 1 -- 172.21.15.40:0/4095320081 <== osd.4 v2:172.21.15.86:6808/150800249 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa364009c20 con 0x7fa35406eb20 2023-10-16T08:38:50.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.616+0000 7fa37229e640 1 -- 172.21.15.40:0/4095320081 >> [v2:172.21.15.86:6808/150800249,v1:172.21.15.86:6809/150800249] conn(0x7fa35406eb20 msgr2=0x7fa354070f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.616+0000 7fa37229e640 1 --2- 172.21.15.40:0/4095320081 >> [v2:172.21.15.86:6808/150800249,v1:172.21.15.86:6809/150800249] conn(0x7fa35406eb20 0x7fa354070f40 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.616+0000 7fa37229e640 1 -- 172.21.15.40:0/4095320081 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa354066a70 msgr2=0x7fa354068f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.616+0000 7fa37229e640 1 --2- 172.21.15.40:0/4095320081 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa354066a70 0x7fa354068f30 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa35c01e700 tx=0x7fa35c008040 comp rx=0 tx=0).stop 2023-10-16T08:38:50.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.616+0000 7fa37229e640 1 -- 172.21.15.40:0/4095320081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa36c0cb370 msgr2=0x7fa36c0c6440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.616+0000 7fa37229e640 1 --2- 172.21.15.40:0/4095320081 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa36c0cb370 0x7fa36c0c6440 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fa36000d5b0 tx=0x7fa360001e80 comp rx=0 tx=0).stop 2023-10-16T08:38:50.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.620+0000 7fa37083b640 1 -- 172.21.15.40:0/4095320081 reap_dead start 2023-10-16T08:38:50.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.620+0000 7fa37229e640 1 -- 172.21.15.40:0/4095320081 shutdown_connections 2023-10-16T08:38:50.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.620+0000 7fa37229e640 1 -- 172.21.15.40:0/4095320081 >> 172.21.15.40:0/4095320081 conn(0x7fa36c0b0470 msgr2=0x7fa36c155210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.620+0000 7fa37229e640 1 -- 172.21.15.40:0/4095320081 shutdown_connections 2023-10-16T08:38:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.620+0000 7fa37229e640 1 -- 172.21.15.40:0/4095320081 wait complete. 2023-10-16T08:38:50.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.620+0000 7fe84131b640 1 --2- 172.21.15.40:0/728071304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe83c0caa90 0x7fe83c08a7f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe834008ec0 tx=0x7fe83401cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8b74156ab6c3968 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.620+0000 7fe83b7fe640 1 -- 172.21.15.40:0/728071304 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8340a5df0 con 0x7fe83c0caa90 2023-10-16T08:38:50.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.620+0000 7fe83b7fe640 1 -- 172.21.15.40:0/728071304 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8340ad440 con 0x7fe83c0caa90 2023-10-16T08:38:50.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.620+0000 7fe83b7fe640 1 -- 172.21.15.40:0/728071304 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8340ad810 con 0x7fe83c0caa90 2023-10-16T08:38:50.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.620+0000 7fe841b1c640 1 -- 172.21.15.40:0/728071304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe83c0caa90 msgr2=0x7fe83c08a7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.620+0000 7fe841b1c640 1 --2- 172.21.15.40:0/728071304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe83c0caa90 0x7fe83c08a7f0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe834008ec0 tx=0x7fe83401cd20 comp rx=0 tx=0).stop 2023-10-16T08:38:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.620+0000 7fe841b1c640 1 -- 172.21.15.40:0/728071304 shutdown_connections 2023-10-16T08:38:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.620+0000 7fe841b1c640 1 --2- 172.21.15.40:0/728071304 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe83c0caa90 0x7fe83c08a7f0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.620+0000 7fe841b1c640 1 --2- 172.21.15.40:0/728071304 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe83c0ca6b0 0x7fe83c0cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.620+0000 7fe841b1c640 1 -- 172.21.15.40:0/728071304 >> 172.21.15.40:0/728071304 conn(0x7fe83c0cb080 msgr2=0x7fe83c0cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.620+0000 7fe841b1c640 1 -- 172.21.15.40:0/728071304 shutdown_connections 2023-10-16T08:38:50.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.620+0000 7fe841b1c640 1 -- 172.21.15.40:0/728071304 wait complete. 2023-10-16T08:38:50.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7f1cfeaea640 1 --2- 172.21.15.40:0/756309401 >> [v2:172.21.15.40:6816/3244406559,v1:172.21.15.40:6818/3244406559] conn(0x7f1cdc06e5a0 0x7f1cdc0709c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7f1cfeaea640 1 --2- 172.21.15.40:0/756309401 >> [v2:172.21.15.40:6816/3244406559,v1:172.21.15.40:6818/3244406559] conn(0x7f1cdc06e5a0 0x7f1cdc0709c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7f1ceeffd640 1 -- 172.21.15.40:0/756309401 <== osd.0 v2:172.21.15.40:6816/3244406559 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f1cdc071070 con 0x7f1cdc06e5a0 2023-10-16T08:38:50.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7fe841b1c640 1 Processor -- start 2023-10-16T08:38:50.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7fe841b1c640 1 -- start start 2023-10-16T08:38:50.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7fe841b1c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe83c071170 0x7fe83c086750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7fe841b1c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe83c0ca6b0 0x7fe83c16b250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7fe841b1c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe83c0caa90 0x7fe83c0af4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7fe83bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe83c0ca6b0 0x7fe83c16b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7fe83bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe83c0ca6b0 0x7fe83c16b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58274/0 (socket says 172.21.15.40:58274) 2023-10-16T08:38:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7fe840b1a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe83c071170 0x7fe83c086750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7fe83bfff640 1 -- 172.21.15.40:0/3139301921 learned_addr learned my addr 172.21.15.40:0/3139301921 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7fe841b1c640 1 -- 172.21.15.40:0/3139301921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c0c6770 con 0x7fe83c0ca6b0 2023-10-16T08:38:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7fe841b1c640 1 -- 172.21.15.40:0/3139301921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c0c74a0 con 0x7fe83c071170 2023-10-16T08:38:50.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7fe841b1c640 1 -- 172.21.15.40:0/3139301921 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83c0c5b60 con 0x7fe83c0caa90 2023-10-16T08:38:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7fe84131b640 1 --2- 172.21.15.40:0/3139301921 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe83c0caa90 0x7fe83c0af4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7fe83bfff640 1 -- 172.21.15.40:0/3139301921 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe83c0caa90 msgr2=0x7fe83c0af4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7fe83bfff640 1 --2- 172.21.15.40:0/3139301921 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe83c0caa90 0x7fe83c0af4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7fe83bfff640 1 -- 172.21.15.40:0/3139301921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe83c071170 msgr2=0x7fe83c086750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7fe83bfff640 1 --2- 172.21.15.40:0/3139301921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe83c071170 0x7fe83c086750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.624+0000 7fe83bfff640 1 -- 172.21.15.40:0/3139301921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe834008b20 con 0x7fe83c0ca6b0 2023-10-16T08:38:50.714 INFO:teuthology.orchestra.run.smithi040.stdout:55834574856 2023-10-16T08:38:50.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.628+0000 7fe83bfff640 1 --2- 172.21.15.40:0/3139301921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe83c0ca6b0 0x7fe83c16b250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe83000c490 tx=0x7fe83000c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.715 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-16T08:38:50.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.628+0000 7fe8397fa640 1 -- 172.21.15.40:0/3139301921 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8300052d0 con 0x7fe83c0ca6b0 2023-10-16T08:38:50.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.628+0000 7fe8397fa640 1 -- 172.21.15.40:0/3139301921 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe830005430 con 0x7fe83c0ca6b0 2023-10-16T08:38:50.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.628+0000 7fe841b1c640 1 -- 172.21.15.40:0/3139301921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83c0c5e40 con 0x7fe83c0ca6b0 2023-10-16T08:38:50.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.628+0000 7f510e4ec640 1 -- 172.21.15.40:0/2357837184 --> [v2:172.21.15.40:6817/2222522452,v1:172.21.15.40:6819/2222522452] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f51080c2b50 con 0x7f51080b2460 2023-10-16T08:38:50.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.628+0000 7f5104ff9640 1 -- 172.21.15.40:0/2357837184 <== osd.1 v2:172.21.15.40:6817/2222522452 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f51080c2b50 con 0x7f51080b2460 2023-10-16T08:38:50.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.628+0000 7fe8397fa640 1 -- 172.21.15.40:0/3139301921 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830012870 con 0x7fe83c0ca6b0 2023-10-16T08:38:50.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.628+0000 7fe841b1c640 1 -- 172.21.15.40:0/3139301921 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe83c0c6400 con 0x7fe83c0ca6b0 2023-10-16T08:38:50.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.628+0000 7fe8397fa640 1 -- 172.21.15.40:0/3139301921 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe830012a90 con 0x7fe83c0ca6b0 2023-10-16T08:38:50.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.628+0000 7fe8397fa640 1 --2- 172.21.15.40:0/3139301921 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe820066570 0x7fe820068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.628+0000 7fe8397fa640 1 -- 172.21.15.40:0/3139301921 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe830083050 con 0x7fe83c0ca6b0 2023-10-16T08:38:50.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.628+0000 7fe841b1c640 1 --2- 172.21.15.40:0/3139301921 >> [v2:172.21.15.86:6813/2566952765,v1:172.21.15.86:6815/2566952765] conn(0x7fe83c0c56b0 0x7fe83c0ae7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.628+0000 7fe841b1c640 1 -- 172.21.15.40:0/3139301921 --> [v2:172.21.15.86:6813/2566952765,v1:172.21.15.86:6815/2566952765] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe83c0c65c0 con 0x7fe83c0c56b0 2023-10-16T08:38:50.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.628+0000 7fe840b1a640 1 --2- 172.21.15.40:0/3139301921 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe820066570 0x7fe820068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.628+0000 7fe84131b640 1 --2- 172.21.15.40:0/3139301921 >> [v2:172.21.15.86:6813/2566952765,v1:172.21.15.86:6815/2566952765] conn(0x7fe83c0c56b0 0x7fe83c0ae7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.632+0000 7f50ee7fc640 1 -- 172.21.15.40:0/2357837184 >> [v2:172.21.15.40:6817/2222522452,v1:172.21.15.40:6819/2222522452] conn(0x7f51080b2460 msgr2=0x7f51080b2840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.632+0000 7f50ee7fc640 1 --2- 172.21.15.40:0/2357837184 >> [v2:172.21.15.40:6817/2222522452,v1:172.21.15.40:6819/2222522452] conn(0x7f51080b2460 0x7f51080b2840 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.632+0000 7f50ee7fc640 1 -- 172.21.15.40:0/2357837184 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f50f4066570 msgr2=0x7f50f4068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.632+0000 7f50ee7fc640 1 --2- 172.21.15.40:0/2357837184 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f50f4066570 0x7f50f4068a30 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f50fc00a9d0 tx=0x7f50fc008040 comp rx=0 tx=0).stop 2023-10-16T08:38:50.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.632+0000 7f50ee7fc640 1 -- 172.21.15.40:0/2357837184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f510815cbd0 msgr2=0x7f51080c8cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.632+0000 7f50ee7fc640 1 --2- 172.21.15.40:0/2357837184 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f510815cbd0 0x7f51080c8cb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f510000c780 tx=0x7f510000cc40 comp rx=0 tx=0).stop 2023-10-16T08:38:50.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.632+0000 7f510ca89640 1 -- 172.21.15.40:0/2357837184 reap_dead start 2023-10-16T08:38:50.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.632+0000 7f50ee7fc640 1 -- 172.21.15.40:0/2357837184 shutdown_connections 2023-10-16T08:38:50.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.632+0000 7f50ee7fc640 1 -- 172.21.15.40:0/2357837184 >> 172.21.15.40:0/2357837184 conn(0x7f51080cc360 msgr2=0x7f5108158740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.632+0000 7fe840b1a640 1 --2- 172.21.15.40:0/3139301921 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe820066570 0x7fe820068a30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe83c04eba0 tx=0x7fe828008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.632+0000 7f50ee7fc640 1 -- 172.21.15.40:0/2357837184 shutdown_connections 2023-10-16T08:38:50.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.632+0000 7f50ee7fc640 1 -- 172.21.15.40:0/2357837184 wait complete. 2023-10-16T08:38:50.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.632+0000 7fe84131b640 1 --2- 172.21.15.40:0/3139301921 >> [v2:172.21.15.86:6813/2566952765,v1:172.21.15.86:6815/2566952765] conn(0x7fe83c0c56b0 0x7fe83c0ae7a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.632+0000 7fe8397fa640 1 -- 172.21.15.40:0/3139301921 <== osd.6 v2:172.21.15.86:6813/2566952765 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fe83c0c65c0 con 0x7fe83c0c56b0 2023-10-16T08:38:50.724 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2023-10-16T08:38:50.725 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-16T08:38:50.726 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-16T08:38:50.726 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-16T08:38:50.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.644+0000 7f1d0054d640 1 -- 172.21.15.40:0/756309401 --> [v2:172.21.15.40:6816/3244406559,v1:172.21.15.40:6818/3244406559] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1cf80cd970 con 0x7f1cdc06e5a0 2023-10-16T08:38:50.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.644+0000 7f1ceeffd640 1 -- 172.21.15.40:0/756309401 <== osd.0 v2:172.21.15.40:6816/3244406559 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1cf00ad6a0 con 0x7f1cdc06e5a0 2023-10-16T08:38:50.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.644+0000 7f1d0054d640 1 -- 172.21.15.40:0/756309401 >> [v2:172.21.15.40:6816/3244406559,v1:172.21.15.40:6818/3244406559] conn(0x7f1cdc06e5a0 msgr2=0x7f1cdc0709c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.644+0000 7f1d0054d640 1 --2- 172.21.15.40:0/756309401 >> [v2:172.21.15.40:6816/3244406559,v1:172.21.15.40:6818/3244406559] conn(0x7f1cdc06e5a0 0x7f1cdc0709c0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.644+0000 7f1d0054d640 1 -- 172.21.15.40:0/756309401 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1cdc066570 msgr2=0x7f1cdc068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.644+0000 7f1d0054d640 1 --2- 172.21.15.40:0/756309401 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1cdc066570 0x7f1cdc068a30 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8006db0 tx=0x7f1ce8009210 comp rx=0 tx=0).stop 2023-10-16T08:38:50.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.644+0000 7f1d0054d640 1 -- 172.21.15.40:0/756309401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cf80caec0 msgr2=0x7f1cf80c46b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.644+0000 7f1d0054d640 1 --2- 172.21.15.40:0/756309401 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1cf80caec0 0x7f1cf80c46b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1cf400e870 tx=0x7f1cf400ed30 comp rx=0 tx=0).stop 2023-10-16T08:38:50.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.644+0000 7f1cfeaea640 1 -- 172.21.15.40:0/756309401 reap_dead start 2023-10-16T08:38:50.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.644+0000 7f1d0054d640 1 -- 172.21.15.40:0/756309401 shutdown_connections 2023-10-16T08:38:50.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.644+0000 7f1d0054d640 1 -- 172.21.15.40:0/756309401 >> 172.21.15.40:0/756309401 conn(0x7f1cf80affc0 msgr2=0x7f1cf80af760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.644+0000 7f1d0054d640 1 -- 172.21.15.40:0/756309401 shutdown_connections 2023-10-16T08:38:50.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.644+0000 7f1d0054d640 1 -- 172.21.15.40:0/756309401 wait complete. 2023-10-16T08:38:50.731 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2023-10-16T08:38:50.731 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-16T08:38:50.732 INFO:teuthology.orchestra.run.smithi040.stdout:55834574856 2023-10-16T08:38:50.733 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-16T08:38:50.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.656+0000 7fe841b1c640 1 -- 172.21.15.40:0/3139301921 --> [v2:172.21.15.86:6813/2566952765,v1:172.21.15.86:6815/2566952765] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe83c067570 con 0x7fe83c0c56b0 2023-10-16T08:38:50.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.656+0000 7fe8397fa640 1 -- 172.21.15.40:0/3139301921 <== osd.6 v2:172.21.15.86:6813/2566952765 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe83c067570 con 0x7fe83c0c56b0 2023-10-16T08:38:50.734 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-16T08:38:50.735 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-16T08:38:50.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.656+0000 7fe841b1c640 1 -- 172.21.15.40:0/3139301921 >> [v2:172.21.15.86:6813/2566952765,v1:172.21.15.86:6815/2566952765] conn(0x7fe83c0c56b0 msgr2=0x7fe83c0ae7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.656+0000 7fe841b1c640 1 --2- 172.21.15.40:0/3139301921 >> [v2:172.21.15.86:6813/2566952765,v1:172.21.15.86:6815/2566952765] conn(0x7fe83c0c56b0 0x7fe83c0ae7a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.656+0000 7fe841b1c640 1 -- 172.21.15.40:0/3139301921 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe820066570 msgr2=0x7fe820068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.656+0000 7fe841b1c640 1 --2- 172.21.15.40:0/3139301921 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe820066570 0x7fe820068a30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fe83c04eba0 tx=0x7fe828008040 comp rx=0 tx=0).stop 2023-10-16T08:38:50.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.656+0000 7fe841b1c640 1 -- 172.21.15.40:0/3139301921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe83c0ca6b0 msgr2=0x7fe83c16b250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.656+0000 7fe841b1c640 1 --2- 172.21.15.40:0/3139301921 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe83c0ca6b0 0x7fe83c16b250 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fe83000c490 tx=0x7fe83000c950 comp rx=0 tx=0).stop 2023-10-16T08:38:50.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.656+0000 7fe84131b640 1 -- 172.21.15.40:0/3139301921 reap_dead start 2023-10-16T08:38:50.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.656+0000 7fe841b1c640 1 -- 172.21.15.40:0/3139301921 shutdown_connections 2023-10-16T08:38:50.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.656+0000 7fe841b1c640 1 -- 172.21.15.40:0/3139301921 >> 172.21.15.40:0/3139301921 conn(0x7fe83c0cb080 msgr2=0x7fe83c0ae010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.656+0000 7fe841b1c640 1 -- 172.21.15.40:0/3139301921 shutdown_connections 2023-10-16T08:38:50.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.656+0000 7fe841b1c640 1 -- 172.21.15.40:0/3139301921 wait complete. 2023-10-16T08:38:50.738 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-16T08:38:50.739 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-16T08:38:50.740 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-16T08:38:50.740 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-16T08:38:50.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.936+0000 7f993376e640 1 Processor -- start 2023-10-16T08:38:50.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.936+0000 7f993376e640 1 -- start start 2023-10-16T08:38:50.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.936+0000 7f993376e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f992c15d3f0 0x7f992c0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.936+0000 7f993376e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f992c15d880 con 0x7f992c0b05d0 2023-10-16T08:38:50.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.936+0000 7f993376e640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f992c06f270 con 0x7f992c158cd0 2023-10-16T08:38:50.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.936+0000 7f993376e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f992c0b0a40 con 0x7f992c15d3f0 2023-10-16T08:38:50.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.940+0000 7f9931d0b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f992c0b05d0 0x7f992c06fa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58362/0 (socket says 172.21.15.40:58362) 2023-10-16T08:38:50.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.940+0000 7f9931d0b640 1 -- 172.21.15.40:0/378803907 learned_addr learned my addr 172.21.15.40:0/378803907 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:50.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.940+0000 7f9930d09640 1 --2- 172.21.15.40:0/378803907 >> v2:172.21.15.40:3300/0 conn(0x7f992c15d3f0 0x7f992c0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.940+0000 7f9930d09640 1 -- 172.21.15.40:0/378803907 >> v1:172.21.15.86:6789/0 conn(0x7f992c158cd0 legacy=0x7f992c06c8a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T08:38:50.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.940+0000 7f9930d09640 1 -- 172.21.15.40:0/378803907 >> v1:172.21.15.40:6789/0 conn(0x7f992c0b05d0 legacy=0x7f992c06fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.940+0000 7f9930d09640 1 -- 172.21.15.40:0/378803907 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992c06ad40 con 0x7f992c15d3f0 2023-10-16T08:38:50.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.940+0000 7f51315fa640 1 Processor -- start 2023-10-16T08:38:50.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.940+0000 7f51315fa640 1 -- start start 2023-10-16T08:38:50.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.940+0000 7f63dffff640 1 Processor -- start 2023-10-16T08:38:50.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.940+0000 7f63dffff640 1 -- start start 2023-10-16T08:38:50.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.940+0000 7fb7ab567640 1 Processor -- start 2023-10-16T08:38:50.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.940+0000 7fb7ab567640 1 -- start start 2023-10-16T08:38:50.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.940+0000 7f51315fa640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f512c158cd0 0x7f512c15d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.940+0000 7f51315fa640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f512c06c8a0 0x7f512c0b0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.940+0000 7f51315fa640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f512c0b05d0 0x7f512c0cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.940+0000 7f51315fa640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512c0b0a60 con 0x7f512c0b05d0 2023-10-16T08:38:50.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.940+0000 7f51315fa640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f512c06f270 con 0x7f512c06c8a0 2023-10-16T08:38:50.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.940+0000 7f51315fa640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512c06ad40 con 0x7f512c158cd0 2023-10-16T08:38:50.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fe017ad8640 1 Processor -- start 2023-10-16T08:38:50.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f9930d09640 1 --2- 172.21.15.40:0/378803907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f992c15d3f0 0x7f992c0b0090 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f992800ba50 tx=0x7f992801e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=757e73492b8db083 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f512b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f512c0b05d0 0x7f512c0cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f512b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f512c0b05d0 0x7f512c0cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58286/0 (socket says 172.21.15.40:58286) 2023-10-16T08:38:50.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f512b7fe640 1 -- 172.21.15.40:0/575289749 learned_addr learned my addr 172.21.15.40:0/575289749 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f512affd640 1 --2- 172.21.15.40:0/575289749 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f512c158cd0 0x7f512c15d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f512a7fc640 1 --2- 172.21.15.40:0/575289749 >> v2:172.21.15.86:3300/0 conn(0x7f512c06c8a0 0x7f512c0b0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63dffff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63d8050330 0x7f63d80531a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.951 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f9923fff640 1 -- 172.21.15.40:0/378803907 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99280a7a20 con 0x7f992c15d3f0 2023-10-16T08:38:50.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f9923fff640 1 -- 172.21.15.40:0/378803907 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99280ab070 con 0x7f992c15d3f0 2023-10-16T08:38:50.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f9923fff640 1 -- 172.21.15.40:0/378803907 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99280a7da0 con 0x7f992c15d3f0 2023-10-16T08:38:50.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fb7ab567640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7a40ad980 0x7fb7a406fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.952 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fb7ab567640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a40701c0 con 0x7fb7a406c650 2023-10-16T08:38:50.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fb7ab567640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a40704a0 con 0x7fb7a40add60 2023-10-16T08:38:50.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fb7ab567640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4070780 con 0x7fb7a40ad980 2023-10-16T08:38:50.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f512b7fe640 1 -- 172.21.15.40:0/575289749 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f512c158cd0 msgr2=0x7f512c15d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.953 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f512b7fe640 1 --2- 172.21.15.40:0/575289749 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f512c158cd0 0x7f512c15d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f512b7fe640 1 -- 172.21.15.40:0/575289749 >> v2:172.21.15.86:3300/0 conn(0x7f512c06c8a0 msgr2=0x7f512c0b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f512b7fe640 1 --2- 172.21.15.40:0/575289749 >> v2:172.21.15.86:3300/0 conn(0x7f512c06c8a0 0x7f512c0b0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f512b7fe640 1 -- 172.21.15.40:0/575289749 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f512c06b020 con 0x7f512c0b05d0 2023-10-16T08:38:50.954 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63deffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f63d80df260 0x7f63d804fc20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58374/0 (socket says 172.21.15.40:58374) 2023-10-16T08:38:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63deffd640 1 -- 172.21.15.40:0/3072336149 learned_addr learned my addr 172.21.15.40:0/3072336149 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fe017ad8640 1 -- start start 2023-10-16T08:38:50.955 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63dffff640 1 -- 172.21.15.40:0/3072336149 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d8054940 con 0x7f63d80df260 2023-10-16T08:38:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f56a5365640 1 Processor -- start 2023-10-16T08:38:50.956 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f56a5365640 1 -- start start 2023-10-16T08:38:50.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fb7a9303640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7a40ad980 0x7fb7a406fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fb7a9303640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7a40ad980 0x7fb7a406fae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.40:54598/0 (socket says 172.21.15.40:54598) 2023-10-16T08:38:50.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fb7a9303640 1 -- 172.21.15.40:0/4085867621 learned_addr learned my addr 172.21.15.40:0/4085867621 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.957 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63de7fc640 1 --2- 172.21.15.40:0/3072336149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63d8050330 0x7f63d80531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63ddffb640 1 -- 172.21.15.40:0/3072336149 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491663373 0 0) 0x7f63d8054940 con 0x7f63d80df260 2023-10-16T08:38:50.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fe017ad8640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe0101591e0 0x7fe01015ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.958 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fe017ad8640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe01006fca0 0x7fe010070130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63dffff640 1 -- 172.21.15.40:0/3072336149 --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63d804f970 con 0x7f63d80e2e40 2023-10-16T08:38:50.959 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63dffff640 1 -- 172.21.15.40:0/3072336149 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d80538a0 con 0x7f63d8050330 2023-10-16T08:38:50.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fe017ad8640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0101595c0 con 0x7fe0100ce7c0 2023-10-16T08:38:50.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fe017ad8640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe01006d210 con 0x7fe01006fca0 2023-10-16T08:38:50.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fe017ad8640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe01015d1f0 con 0x7fe0101591e0 2023-10-16T08:38:50.960 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f993376e640 1 -- 172.21.15.40:0/378803907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f992c15d3f0 msgr2=0x7f992c0b0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f993376e640 1 --2- 172.21.15.40:0/378803907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f992c15d3f0 0x7f992c0b0090 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f992800ba50 tx=0x7f992801e970 comp rx=0 tx=0).stop 2023-10-16T08:38:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fb7a9303640 1 -- 172.21.15.40:0/4085867621 >> v1:172.21.15.86:6789/0 conn(0x7fb7a40add60 legacy=0x7fb7a40cbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fb7a9303640 1 -- 172.21.15.40:0/4085867621 >> v1:172.21.15.40:6789/0 conn(0x7fb7a406c650 legacy=0x7fb7a40c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.961 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fb7a9303640 1 -- 172.21.15.40:0/4085867621 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a4070a60 con 0x7fb7a40ad980 2023-10-16T08:38:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f9a62feb640 1 Processor -- start 2023-10-16T08:38:50.962 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63ddffb640 1 -- 172.21.15.40:0/3072336149 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63c8003530 con 0x7f63d80df260 2023-10-16T08:38:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63ddffb640 1 -- 172.21.15.40:0/3072336149 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 66943188 0 0) 0x7f63e0068670 con 0x7f63d80e2e40 2023-10-16T08:38:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63ddffb640 1 -- 172.21.15.40:0/3072336149 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63d8054940 con 0x7f63d80e2e40 2023-10-16T08:38:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fe015874640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe0101591e0 0x7fe01015ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.963 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fe015874640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe0101591e0 0x7fe01015ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.40:53918/0 (socket says 172.21.15.40:53918) 2023-10-16T08:38:50.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fe015874640 1 -- 172.21.15.40:0/1159740278 learned_addr learned my addr 172.21.15.40:0/1159740278 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f993376e640 1 -- 172.21.15.40:0/378803907 shutdown_connections 2023-10-16T08:38:50.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f993376e640 1 --2- 172.21.15.40:0/378803907 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f992c15d3f0 0x7f992c0b0090 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.964 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f993376e640 1 -- 172.21.15.40:0/378803907 >> 172.21.15.40:0/378803907 conn(0x7f992c0cb090 msgr2=0x7f992c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f993376e640 1 -- 172.21.15.40:0/378803907 shutdown_connections 2023-10-16T08:38:50.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f993376e640 1 -- 172.21.15.40:0/378803907 wait complete. 2023-10-16T08:38:50.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63ddffb640 1 -- 172.21.15.40:0/3072336149 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2265336629 0 0) 0x7f63c8003530 con 0x7f63d80df260 2023-10-16T08:38:50.965 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63ddffb640 1 -- 172.21.15.40:0/3072336149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63d8050330 msgr2=0x7f63d80531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63ddffb640 1 --2- 172.21.15.40:0/3072336149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63d8050330 0x7f63d80531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63ddffb640 1 -- 172.21.15.40:0/3072336149 >> v1:172.21.15.86:6789/0 conn(0x7f63d80e2e40 legacy=0x7f63d8005ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63ddffb640 1 -- 172.21.15.40:0/3072336149 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d8053b80 con 0x7f63d80df260 2023-10-16T08:38:50.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63ddffb640 1 -- 172.21.15.40:0/3072336149 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f63d0004cc0 con 0x7f63d80df260 2023-10-16T08:38:50.966 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63ddffb640 1 -- 172.21.15.40:0/3072336149 >> v1:172.21.15.40:6789/0 conn(0x7f63d80df260 legacy=0x7f63d804fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63ddffb640 1 --2- 172.21.15.40:0/3072336149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63c8003b40 0x7f63c8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63ddffb640 1 --2- 172.21.15.40:0/3072336149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63c80047d0 0x7f63c8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63ddffb640 1 --2- 172.21.15.40:0/3072336149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63c8005190 0x7f63c80058d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63ddffb640 1 -- 172.21.15.40:0/3072336149 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d80538a0 con 0x7f63c80047d0 2023-10-16T08:38:50.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63ddffb640 1 -- 172.21.15.40:0/3072336149 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c8006020 con 0x7f63c8003b40 2023-10-16T08:38:50.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63ddffb640 1 -- 172.21.15.40:0/3072336149 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c80061e0 con 0x7f63c8005190 2023-10-16T08:38:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63df7fe640 1 --2- 172.21.15.40:0/3072336149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63c8005190 0x7f63c80058d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f56a5365640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f56a015cf40 0x7f56a00ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f56a5365640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f56a015d320 0x7f56a006f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.968 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f56a5365640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56a00ade00 0x7f56a00ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f56a5365640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56a006b540 con 0x7f56a015cf40 2023-10-16T08:38:50.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f56a5365640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56a006b7e0 con 0x7f56a015d320 2023-10-16T08:38:50.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f56a5365640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a006bac0 con 0x7f56a00ade00 2023-10-16T08:38:50.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f569f7fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56a00ade00 0x7f56a00ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.969 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f569f7fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56a00ade00 0x7f56a00ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.40:54602/0 (socket says 172.21.15.40:54602) 2023-10-16T08:38:50.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f569f7fe640 1 -- 172.21.15.40:0/419916624 learned_addr learned my addr 172.21.15.40:0/419916624 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fe016075640 1 --2- 172.21.15.40:0/1159740278 >> v2:172.21.15.40:3300/0 conn(0x7fe01006fca0 0x7fe010070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fe015874640 1 -- 172.21.15.40:0/1159740278 >> v2:172.21.15.40:3300/0 conn(0x7fe01006fca0 msgr2=0x7fe010070130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fe015874640 1 --2- 172.21.15.40:0/1159740278 >> v2:172.21.15.40:3300/0 conn(0x7fe01006fca0 0x7fe010070130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fe015874640 1 -- 172.21.15.40:0/1159740278 >> v1:172.21.15.86:6789/0 conn(0x7fe0100ce7c0 legacy=0x7fe01006d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7fe015874640 1 -- 172.21.15.40:0/1159740278 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe01015d490 con 0x7fe0101591e0 2023-10-16T08:38:50.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f1f299a8640 1 Processor -- start 2023-10-16T08:38:50.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f1f299a8640 1 -- start start 2023-10-16T08:38:50.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63deffd640 1 --2- 172.21.15.40:0/3072336149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63c8003b40 0x7f63c8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f9a62feb640 1 -- start start 2023-10-16T08:38:50.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f1f299a8640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1f2415b6e0 0x7f1f241626d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f9a62feb640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f9a5c0adb80 0x7f9a5c159e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f9a62feb640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9a5c0cb370 0x7f9a5c0cb7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f9a62feb640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a5c0ae0d0 con 0x7f9a5c156a20 2023-10-16T08:38:50.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f9a62feb640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a5c0cb0a0 con 0x7f9a5c0adb80 2023-10-16T08:38:50.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f9a62feb640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a5c156e00 con 0x7f9a5c0cb370 2023-10-16T08:38:50.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f1f227fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1f24162c10 0x7f1f240705d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58384/0 (socket says 172.21.15.40:58384) 2023-10-16T08:38:50.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f1f227fc640 1 -- 172.21.15.40:0/1115352663 learned_addr learned my addr 172.21.15.40:0/1115352663 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:50.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f1f299a8640 1 --2- 172.21.15.40:0/1115352663 >> v2:172.21.15.40:3300/0 conn(0x7f1f2406d590 0x7f1f2406d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f569f7fe640 1 -- 172.21.15.40:0/419916624 >> v2:172.21.15.86:3300/0 conn(0x7f56a015d320 msgr2=0x7f56a006f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f569f7fe640 1 --2- 172.21.15.40:0/419916624 >> v2:172.21.15.86:3300/0 conn(0x7f56a015d320 0x7f56a006f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f569f7fe640 1 -- 172.21.15.40:0/419916624 >> v2:172.21.15.40:3300/0 conn(0x7f56a015cf40 msgr2=0x7f56a00ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f569f7fe640 1 --2- 172.21.15.40:0/419916624 >> v2:172.21.15.40:3300/0 conn(0x7f56a015cf40 0x7f56a00ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f569f7fe640 1 -- 172.21.15.40:0/419916624 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a006bda0 con 0x7f56a00ade00 2023-10-16T08:38:50.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f9a61588640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9a5c0cb370 0x7f9a5c0cb7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f9a61588640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9a5c0cb370 0x7f9a5c0cb7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.40:53950/0 (socket says 172.21.15.40:53950) 2023-10-16T08:38:50.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f9a61588640 1 -- 172.21.15.40:0/1620497622 learned_addr learned my addr 172.21.15.40:0/1620497622 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63df7fe640 1 -- 172.21.15.40:0/3072336149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63c8003b40 msgr2=0x7f63c8003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63df7fe640 1 --2- 172.21.15.40:0/3072336149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63c8003b40 0x7f63c8003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63df7fe640 1 -- 172.21.15.40:0/3072336149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63c80047d0 msgr2=0x7f63c8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63df7fe640 1 --2- 172.21.15.40:0/3072336149 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63c80047d0 0x7f63c8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.944+0000 7f63df7fe640 1 -- 172.21.15.40:0/3072336149 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d8053b80 con 0x7f63c8005190 2023-10-16T08:38:50.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f1f299a8640 1 -- 172.21.15.40:0/1115352663 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f2406def0 con 0x7f1f24162c10 2023-10-16T08:38:50.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f1f299a8640 1 -- 172.21.15.40:0/1115352663 --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f2406f5c0 con 0x7f1f2406d590 2023-10-16T08:38:50.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f1f299a8640 1 -- 172.21.15.40:0/1115352663 --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f2406f810 con 0x7f1f2415b6e0 2023-10-16T08:38:50.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f1f22ffd640 1 --2- 172.21.15.40:0/1115352663 >> v2:172.21.15.86:3300/0 conn(0x7f1f2415b6e0 0x7f1f241626d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f993376e640 1 Processor -- start 2023-10-16T08:38:50.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f993376e640 1 -- start start 2023-10-16T08:38:50.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f993376e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f992c0b05d0 0x7f992c0aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f993376e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f992c158cd0 0x7f992c0b39d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f993376e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f992c15d3f0 0x7f992c1e6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:50.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f993376e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c0afc40 con 0x7f992c0b05d0 2023-10-16T08:38:50.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f993376e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f992c15ad00 con 0x7f992c158cd0 2023-10-16T08:38:50.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f993376e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f992c15afe0 con 0x7f992c15d3f0 2023-10-16T08:38:50.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f9931d0b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f992c15d3f0 0x7f992c1e6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f993150a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f992c0b05d0 0x7f992c0aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f9931d0b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f992c15d3f0 0x7f992c1e6e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.40:54610/0 (socket says 172.21.15.40:54610) 2023-10-16T08:38:50.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f993150a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f992c0b05d0 0x7f992c0aea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58354/0 (socket says 172.21.15.40:58354) 2023-10-16T08:38:50.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f9931d0b640 1 -- 172.21.15.40:0/1377437207 learned_addr learned my addr 172.21.15.40:0/1377437207 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:50.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f9930d09640 1 --2- 172.21.15.40:0/1377437207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f992c158cd0 0x7f992c0b39d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f9a61588640 1 -- 172.21.15.40:0/1620497622 >> v2:172.21.15.40:3300/0 conn(0x7f9a5c0adb80 msgr2=0x7f9a5c159e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f9a61588640 1 --2- 172.21.15.40:0/1620497622 >> v2:172.21.15.40:3300/0 conn(0x7f9a5c0adb80 0x7f9a5c159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f9a61588640 1 -- 172.21.15.40:0/1620497622 >> v1:172.21.15.40:6789/0 conn(0x7f9a5c156a20 legacy=0x7f9a5c0cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f9a61588640 1 -- 172.21.15.40:0/1620497622 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5c0ccc40 con 0x7f9a5c0cb370 2023-10-16T08:38:50.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7fb7a9303640 1 --2- 172.21.15.40:0/4085867621 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7a40ad980 0x7fb7a406fae0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00098a0 tx=0x7fb7a001c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1df7b34f6a5be3df server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7fb79bfff640 1 -- 172.21.15.40:0/4085867621 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a00aa610 con 0x7fb7a40ad980 2023-10-16T08:38:50.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7fb79bfff640 1 -- 172.21.15.40:0/4085867621 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a00a6040 con 0x7fb7a40ad980 2023-10-16T08:38:50.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7fb79bfff640 1 -- 172.21.15.40:0/4085867621 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a00aa990 con 0x7fb7a40ad980 2023-10-16T08:38:50.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7fb7ab567640 1 -- 172.21.15.40:0/4085867621 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7a40ad980 msgr2=0x7fb7a406fae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7fb7ab567640 1 --2- 172.21.15.40:0/4085867621 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7a40ad980 0x7fb7a406fae0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00098a0 tx=0x7fb7a001c4f0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f512b7fe640 1 --2- 172.21.15.40:0/575289749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f512c0b05d0 0x7f512c0cc510 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f512c06e620 tx=0x7f511c01ea10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28e78d34506901b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f5129ffb640 1 -- 172.21.15.40:0/575289749 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c0a78a0 con 0x7f512c0b05d0 2023-10-16T08:38:50.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f5129ffb640 1 -- 172.21.15.40:0/575289749 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f511c0ab070 con 0x7f512c0b05d0 2023-10-16T08:38:50.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f5129ffb640 1 -- 172.21.15.40:0/575289749 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511c0a7c20 con 0x7f512c0b05d0 2023-10-16T08:38:50.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7fe015874640 1 --2- 172.21.15.40:0/1159740278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0101591e0 0x7fe01015ccb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe010066a60 tx=0x7fe00c01c490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9ad075796c9a79f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7fe014872640 1 -- 172.21.15.40:0/1159740278 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c0ab600 con 0x7fe0101591e0 2023-10-16T08:38:50.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7fe014872640 1 -- 172.21.15.40:0/1159740278 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe00c0a7050 con 0x7fe0101591e0 2023-10-16T08:38:50.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7fe014872640 1 -- 172.21.15.40:0/1159740278 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe00c0ab980 con 0x7fe0101591e0 2023-10-16T08:38:50.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7fe017ad8640 1 -- 172.21.15.40:0/1159740278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0101591e0 msgr2=0x7fe01015ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7fe017ad8640 1 --2- 172.21.15.40:0/1159740278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0101591e0 0x7fe01015ccb0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fe010066a60 tx=0x7fe00c01c490 comp rx=0 tx=0).stop 2023-10-16T08:38:50.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7fe017ad8640 1 -- 172.21.15.40:0/1159740278 shutdown_connections 2023-10-16T08:38:50.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7fe017ad8640 1 --2- 172.21.15.40:0/1159740278 >> v2:172.21.15.40:3300/0 conn(0x7fe01006fca0 0x7fe010070130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7fe017ad8640 1 --2- 172.21.15.40:0/1159740278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0101591e0 0x7fe01015ccb0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7fe017ad8640 1 -- 172.21.15.40:0/1159740278 >> 172.21.15.40:0/1159740278 conn(0x7fe0100b0be0 msgr2=0x7fe0100b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7fe017ad8640 1 -- 172.21.15.40:0/1159740278 shutdown_connections 2023-10-16T08:38:50.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7fe017ad8640 1 -- 172.21.15.40:0/1159740278 wait complete. 2023-10-16T08:38:50.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f1f237fe640 1 --2- 172.21.15.40:0/1115352663 >> v2:172.21.15.40:3300/0 conn(0x7f1f2406d590 0x7f1f2406d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:50.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f1f237fe640 1 -- 172.21.15.40:0/1115352663 >> v2:172.21.15.86:3300/0 conn(0x7f1f2415b6e0 msgr2=0x7f1f241626d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:50.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f1f237fe640 1 --2- 172.21.15.40:0/1115352663 >> v2:172.21.15.86:3300/0 conn(0x7f1f2415b6e0 0x7f1f241626d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f1f237fe640 1 -- 172.21.15.40:0/1115352663 >> v1:172.21.15.40:6789/0 conn(0x7f1f24162c10 legacy=0x7f1f240705d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.992 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f1f237fe640 1 -- 172.21.15.40:0/1115352663 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f2406faf0 con 0x7f1f2406d590 2023-10-16T08:38:50.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f569f7fe640 1 --2- 172.21.15.40:0/419916624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56a00ade00 0x7f56a00ac8e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f56a00725b0 tx=0x7f569401c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e805f1901c6afc2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f63df7fe640 1 --2- 172.21.15.40:0/3072336149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63c8005190 0x7f63c80058d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f63e0068a60 tx=0x7f63e0096600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e7828cdcb9e9d0c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.993 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f9931d0b640 1 -- 172.21.15.40:0/1377437207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f992c158cd0 msgr2=0x7f992c0b39d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f9931d0b640 1 --2- 172.21.15.40:0/1377437207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f992c158cd0 0x7f992c0b39d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f9931d0b640 1 -- 172.21.15.40:0/1377437207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f992c0b05d0 msgr2=0x7f992c0aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f9931d0b640 1 --2- 172.21.15.40:0/1377437207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f992c0b05d0 0x7f992c0aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.994 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f9931d0b640 1 -- 172.21.15.40:0/1377437207 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f992800b6b0 con 0x7f992c15d3f0 2023-10-16T08:38:50.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f9a61588640 1 --2- 172.21.15.40:0/1620497622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c0cb370 0x7f9a5c0cb7e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c0cb2f0 tx=0x7f9a5401ed50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d1bee5e950f9f1a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f569dffb640 1 -- 172.21.15.40:0/419916624 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56940ac600 con 0x7f56a00ade00 2023-10-16T08:38:50.995 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f9931d0b640 1 --2- 172.21.15.40:0/1377437207 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f992c15d3f0 0x7f992c1e6e40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f992400f780 tx=0x7f992400fc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:50.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f569dffb640 1 -- 172.21.15.40:0/419916624 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56940a8050 con 0x7f56a00ade00 2023-10-16T08:38:50.996 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f569dffb640 1 -- 172.21.15.40:0/419916624 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56940ac980 con 0x7f56a00ade00 2023-10-16T08:38:50.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f51315fa640 1 -- 172.21.15.40:0/575289749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f512c0b05d0 msgr2=0x7f512c0cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:50.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f51315fa640 1 --2- 172.21.15.40:0/575289749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f512c0b05d0 0x7f512c0cc510 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f512c06e620 tx=0x7f511c01ea10 comp rx=0 tx=0).stop 2023-10-16T08:38:50.997 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.948+0000 7f63ddffb640 1 -- 172.21.15.40:0/3072336149 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e006aea0 con 0x7f63c8005190 2023-10-16T08:38:50.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f51315fa640 1 -- 172.21.15.40:0/575289749 shutdown_connections 2023-10-16T08:38:50.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f51315fa640 1 --2- 172.21.15.40:0/575289749 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f512c0b05d0 0x7f512c0cc510 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f51315fa640 1 --2- 172.21.15.40:0/575289749 >> v2:172.21.15.86:3300/0 conn(0x7f512c06c8a0 0x7f512c0b0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.998 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f51315fa640 1 --2- 172.21.15.40:0/575289749 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f512c158cd0 0x7f512c15d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:50.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f51315fa640 1 -- 172.21.15.40:0/575289749 >> 172.21.15.40:0/575289749 conn(0x7f512c0cb090 msgr2=0x7f512c0cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:50.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f9921ffb640 1 -- 172.21.15.40:0/1377437207 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992400a270 con 0x7f992c15d3f0 2023-10-16T08:38:50.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f9921ffb640 1 -- 172.21.15.40:0/1377437207 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f992400a3d0 con 0x7f992c15d3f0 2023-10-16T08:38:50.999 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f9921ffb640 1 -- 172.21.15.40:0/1377437207 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f992400d840 con 0x7f992c15d3f0 2023-10-16T08:38:51.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f9a5b7fe640 1 -- 172.21.15.40:0/1620497622 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a540a7830 con 0x7f9a5c0cb370 2023-10-16T08:38:51.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f9a5b7fe640 1 -- 172.21.15.40:0/1620497622 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a540ab070 con 0x7f9a5c0cb370 2023-10-16T08:38:51.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f9a5b7fe640 1 -- 172.21.15.40:0/1620497622 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a540a7bb0 con 0x7f9a5c0cb370 2023-10-16T08:38:51.000 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f9a62feb640 1 -- 172.21.15.40:0/1620497622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c0cb370 msgr2=0x7f9a5c0cb7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f9a62feb640 1 --2- 172.21.15.40:0/1620497622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c0cb370 0x7f9a5c0cb7e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9a5c0cb2f0 tx=0x7f9a5401ed50 comp rx=0 tx=0).stop 2023-10-16T08:38:51.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fb7ab567640 1 -- 172.21.15.40:0/4085867621 shutdown_connections 2023-10-16T08:38:51.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fb7ab567640 1 --2- 172.21.15.40:0/4085867621 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7a40ad980 0x7fb7a406fae0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.001 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fb7ab567640 1 -- 172.21.15.40:0/4085867621 >> 172.21.15.40:0/4085867621 conn(0x7fb7a40b0470 msgr2=0x7fb7a40b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:51.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fb7ab567640 1 -- 172.21.15.40:0/4085867621 shutdown_connections 2023-10-16T08:38:51.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f63ddffb640 1 -- 172.21.15.40:0/3072336149 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e0125660 con 0x7f63c8005190 2023-10-16T08:38:51.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f63ddffb640 1 -- 172.21.15.40:0/3072336149 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e0125a50 con 0x7f63c8005190 2023-10-16T08:38:51.002 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f63dffff640 1 -- 172.21.15.40:0/3072336149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63c8005190 msgr2=0x7f63c80058d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f63dffff640 1 --2- 172.21.15.40:0/3072336149 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63c8005190 0x7f63c80058d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f63e0068a60 tx=0x7f63e0096600 comp rx=0 tx=0).stop 2023-10-16T08:38:51.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f56a5365640 1 -- 172.21.15.40:0/419916624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56a00ade00 msgr2=0x7f56a00ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f56a5365640 1 --2- 172.21.15.40:0/419916624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56a00ade00 0x7f56a00ac8e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f56a00725b0 tx=0x7f569401c4f0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f56a5365640 1 -- 172.21.15.40:0/419916624 shutdown_connections 2023-10-16T08:38:51.003 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f56a5365640 1 --2- 172.21.15.40:0/419916624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56a00ade00 0x7f56a00ac8e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f56a5365640 1 --2- 172.21.15.40:0/419916624 >> v2:172.21.15.86:3300/0 conn(0x7f56a015d320 0x7f56a006f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f56a5365640 1 --2- 172.21.15.40:0/419916624 >> v2:172.21.15.40:3300/0 conn(0x7f56a015cf40 0x7f56a00ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f56a5365640 1 -- 172.21.15.40:0/419916624 >> 172.21.15.40:0/419916624 conn(0x7f56a00ce960 msgr2=0x7f56a00ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:51.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f56a5365640 1 -- 172.21.15.40:0/419916624 shutdown_connections 2023-10-16T08:38:51.004 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f56a5365640 1 -- 172.21.15.40:0/419916624 wait complete. 2023-10-16T08:38:51.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f51315fa640 1 -- 172.21.15.40:0/575289749 shutdown_connections 2023-10-16T08:38:51.005 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fe017ad8640 1 Processor -- start 2023-10-16T08:38:51.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f1f237fe640 1 --2- 172.21.15.40:0/1115352663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2406d590 0x7f1f2406d9b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1f24067000 tx=0x7f1f1401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=694f03a9dfb8c168 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:51.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f1f21ffb640 1 -- 172.21.15.40:0/1115352663 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f140ab600 con 0x7f1f2406d590 2023-10-16T08:38:51.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f1f21ffb640 1 -- 172.21.15.40:0/1115352663 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f140a7050 con 0x7f1f2406d590 2023-10-16T08:38:51.006 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f1f21ffb640 1 -- 172.21.15.40:0/1115352663 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f140ab980 con 0x7f1f2406d590 2023-10-16T08:38:51.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f51315fa640 1 -- 172.21.15.40:0/575289749 wait complete. 2023-10-16T08:38:51.007 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f51315fa640 1 Processor -- start 2023-10-16T08:38:51.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f63df7fe640 1 -- 172.21.15.40:0/3072336149 reap_dead start 2023-10-16T08:38:51.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f63dffff640 1 -- 172.21.15.40:0/3072336149 shutdown_connections 2023-10-16T08:38:51.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f63dffff640 1 -- 172.21.15.40:0/3072336149 >> 172.21.15.40:0/3072336149 conn(0x7f63d8054bc0 msgr2=0x7f63d8054fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:51.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f63dffff640 1 -- 172.21.15.40:0/3072336149 shutdown_connections 2023-10-16T08:38:51.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f9a62feb640 1 -- 172.21.15.40:0/1620497622 shutdown_connections 2023-10-16T08:38:51.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f9a62feb640 1 --2- 172.21.15.40:0/1620497622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c0cb370 0x7f9a5c0cb7e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f9a62feb640 1 --2- 172.21.15.40:0/1620497622 >> v2:172.21.15.40:3300/0 conn(0x7f9a5c0adb80 0x7f9a5c159e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f9a62feb640 1 -- 172.21.15.40:0/1620497622 >> 172.21.15.40:0/1620497622 conn(0x7f9a5c0b0470 msgr2=0x7f9a5c0b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:51.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f9a62feb640 1 -- 172.21.15.40:0/1620497622 shutdown_connections 2023-10-16T08:38:51.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f9a62feb640 1 -- 172.21.15.40:0/1620497622 wait complete. 2023-10-16T08:38:51.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fb7ab567640 1 -- 172.21.15.40:0/4085867621 wait complete. 2023-10-16T08:38:51.010 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fb7ab567640 1 Processor -- start 2023-10-16T08:38:51.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fe017ad8640 1 -- start start 2023-10-16T08:38:51.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fe017ad8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe01006fca0 0x7fe010158500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fe017ad8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0100ce7c0 0x7fe01015b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fe017ad8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0101591e0 0x7fe01015bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fe017ad8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100c2ba0 con 0x7fe0101591e0 2023-10-16T08:38:51.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fe017ad8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100c2e80 con 0x7fe0100ce7c0 2023-10-16T08:38:51.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fe017ad8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0100c9190 con 0x7fe01006fca0 2023-10-16T08:38:51.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fe015073640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0100ce7c0 0x7fe01015b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fe015073640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0100ce7c0 0x7fe01015b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.40:53966/0 (socket says 172.21.15.40:53966) 2023-10-16T08:38:51.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fe015073640 1 -- 172.21.15.40:0/2157627838 learned_addr learned my addr 172.21.15.40:0/2157627838 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:51.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fe015874640 1 --2- 172.21.15.40:0/2157627838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe01006fca0 0x7fe010158500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f1f299a8640 1 -- 172.21.15.40:0/1115352663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2406d590 msgr2=0x7f1f2406d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f1f299a8640 1 --2- 172.21.15.40:0/1115352663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2406d590 0x7f1f2406d9b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f1f24067000 tx=0x7f1f1401c460 comp rx=0 tx=0).stop 2023-10-16T08:38:51.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f51315fa640 1 -- start start 2023-10-16T08:38:51.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f1f299a8640 1 -- 172.21.15.40:0/1115352663 shutdown_connections 2023-10-16T08:38:51.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f1f299a8640 1 --2- 172.21.15.40:0/1115352663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2406d590 0x7f1f2406d9b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f1f299a8640 1 --2- 172.21.15.40:0/1115352663 >> v2:172.21.15.86:3300/0 conn(0x7f1f2415b6e0 0x7f1f241626d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f51315fa640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f512c06c8a0 0x7f512c0c2890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f51315fa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f512c0b05d0 0x7f512c0a84d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f51315fa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f512c158cd0 0x7f512c205d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.015 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f51315fa640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0ab700 con 0x7f512c0b05d0 2023-10-16T08:38:51.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f51315fa640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f512c155d10 con 0x7f512c158cd0 2023-10-16T08:38:51.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f51315fa640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f512c0a8a10 con 0x7f512c06c8a0 2023-10-16T08:38:51.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f1f299a8640 1 -- 172.21.15.40:0/1115352663 >> 172.21.15.40:0/1115352663 conn(0x7f1f240661e0 msgr2=0x7f1f240b06a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:51.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f1f299a8640 1 -- 172.21.15.40:0/1115352663 shutdown_connections 2023-10-16T08:38:51.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f512a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f512c0b05d0 0x7f512c0a84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f512a7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f512c0b05d0 0x7f512c0a84d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58366/0 (socket says 172.21.15.40:58366) 2023-10-16T08:38:51.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f512a7fc640 1 -- 172.21.15.40:0/795896490 learned_addr learned my addr 172.21.15.40:0/795896490 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:51.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f512affd640 1 --2- 172.21.15.40:0/795896490 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f512c06c8a0 0x7f512c0c2890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f1f299a8640 1 -- 172.21.15.40:0/1115352663 wait complete. 2023-10-16T08:38:51.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f56a5365640 1 Processor -- start 2023-10-16T08:38:51.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f56a5365640 1 -- start start 2023-10-16T08:38:51.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fe015073640 1 -- 172.21.15.40:0/2157627838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe01006fca0 msgr2=0x7fe010158500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fe015073640 1 --2- 172.21.15.40:0/2157627838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe01006fca0 0x7fe010158500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fe015073640 1 -- 172.21.15.40:0/2157627838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0101591e0 msgr2=0x7fe01015bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fe015073640 1 --2- 172.21.15.40:0/2157627838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0101591e0 0x7fe01015bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fe015073640 1 -- 172.21.15.40:0/2157627838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe00c0094d0 con 0x7fe0100ce7c0 2023-10-16T08:38:51.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fe015874640 1 --2- 172.21.15.40:0/2157627838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe01006fca0 0x7fe010158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:38:51.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f512b7fe640 1 --2- 172.21.15.40:0/795896490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f512c158cd0 0x7f512c205d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fe015073640 1 --2- 172.21.15.40:0/2157627838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0100ce7c0 0x7fe01015b7c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe00000c450 tx=0x7fe00000c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:51.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fb7ab567640 1 -- start start 2023-10-16T08:38:51.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fb7ab567640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a406c650 0x7fb7a42130c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fb7ab567640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7a40ad980 0x7fb7a40c87f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fb7ab567640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7a40add60 0x7fb7a4089b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fb7ab567640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a42086c0 con 0x7fb7a40add60 2023-10-16T08:38:51.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fb7ab567640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a40c8d30 con 0x7fb7a406c650 2023-10-16T08:38:51.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fb7ab567640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a42136f0 con 0x7fb7a40ad980 2023-10-16T08:38:51.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f9a62feb640 1 Processor -- start 2023-10-16T08:38:51.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f63dffff640 1 -- 172.21.15.40:0/3072336149 wait complete. 2023-10-16T08:38:51.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f63dffff640 1 Processor -- start 2023-10-16T08:38:51.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7fe0067fc640 1 -- 172.21.15.40:0/2157627838 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000012830 con 0x7fe0100ce7c0 2023-10-16T08:38:51.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f993376e640 1 -- 172.21.15.40:0/1377437207 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f992c15b2c0 con 0x7f992c15d3f0 2023-10-16T08:38:51.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.952+0000 7f993376e640 1 -- 172.21.15.40:0/1377437207 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f992c15b880 con 0x7f992c15d3f0 2023-10-16T08:38:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fb7a8b02640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7a40ad980 0x7fb7a40c87f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fb7a9303640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a406c650 0x7fb7a42130c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fb7a8b02640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7a40ad980 0x7fb7a40c87f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.40:54648/0 (socket says 172.21.15.40:54648) 2023-10-16T08:38:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fb7a8b02640 1 -- 172.21.15.40:0/358879601 learned_addr learned my addr 172.21.15.40:0/358879601 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:51.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fb7a9b04640 1 --2- 172.21.15.40:0/358879601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7a40add60 0x7fb7a4089b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fb7a9b04640 1 -- 172.21.15.40:0/358879601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7a40ad980 msgr2=0x7fb7a40c87f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fb7a9b04640 1 --2- 172.21.15.40:0/358879601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7a40ad980 0x7fb7a40c87f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fb7a9b04640 1 -- 172.21.15.40:0/358879601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a406c650 msgr2=0x7fb7a42130c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fb7a9b04640 1 --2- 172.21.15.40:0/358879601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a406c650 0x7fb7a42130c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fb7a9b04640 1 -- 172.21.15.40:0/358879601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a00094d0 con 0x7fb7a40add60 2023-10-16T08:38:51.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f56a5365640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56a00ade00 0x7f56a014c9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f56a5365640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56a015cf40 0x7f56a01601a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f56a5365640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a015d320 0x7f56a0155ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f56a5365640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a00ca460 con 0x7f56a015cf40 2023-10-16T08:38:51.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f56a5365640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56a015e400 con 0x7f56a015d320 2023-10-16T08:38:51.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f56a5365640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56a01606e0 con 0x7f56a00ade00 2023-10-16T08:38:51.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f569f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a015d320 0x7f56a0155ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f569f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a015d320 0x7f56a0155ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.40:53986/0 (socket says 172.21.15.40:53986) 2023-10-16T08:38:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f569f7fe640 1 -- 172.21.15.40:0/2993798617 learned_addr learned my addr 172.21.15.40:0/2993798617 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f569e7fc640 1 --2- 172.21.15.40:0/2993798617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56a015cf40 0x7f56a01601a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f512a7fc640 1 -- 172.21.15.40:0/795896490 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f512c06c8a0 msgr2=0x7f512c0c2890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f512a7fc640 1 --2- 172.21.15.40:0/795896490 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f512c06c8a0 0x7f512c0c2890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f512a7fc640 1 -- 172.21.15.40:0/795896490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f512c158cd0 msgr2=0x7f512c205d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f512a7fc640 1 --2- 172.21.15.40:0/795896490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f512c158cd0 0x7f512c205d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f512a7fc640 1 -- 172.21.15.40:0/795896490 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f511c008550 con 0x7f512c0b05d0 2023-10-16T08:38:51.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f512a7fc640 1 --2- 172.21.15.40:0/795896490 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f512c0b05d0 0x7f512c0a84d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f512000d830 tx=0x7f512000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:51.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f51137fe640 1 -- 172.21.15.40:0/795896490 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f512000be10 con 0x7f512c0b05d0 2023-10-16T08:38:51.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f51137fe640 1 -- 172.21.15.40:0/795896490 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51200098b0 con 0x7f512c0b05d0 2023-10-16T08:38:51.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f51137fe640 1 -- 172.21.15.40:0/795896490 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5120009a20 con 0x7f512c0b05d0 2023-10-16T08:38:51.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f9a62feb640 1 -- start start 2023-10-16T08:38:51.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f9a62feb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c0adb80 0x7f9a5c14cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f9a62feb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a5c0cb370 0x7f9a5c14f200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f9a62feb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9a5c156a20 0x7f9a5c153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f9a62feb640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c155c30 con 0x7f9a5c0cb370 2023-10-16T08:38:51.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f9a62feb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c155f10 con 0x7f9a5c0adb80 2023-10-16T08:38:51.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f9a62feb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a5c1561f0 con 0x7f9a5c156a20 2023-10-16T08:38:51.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f63dffff640 1 -- start start 2023-10-16T08:38:51.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f63dffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63d8188720 0x7f63d804e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f63dffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d80de190 0x7f63d80de610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f63dffff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63d80ed5c0 0x7f63d80eda40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f63dffff640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d804dd40 con 0x7f63d80de190 2023-10-16T08:38:51.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f63dffff640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63d80edf80 con 0x7f63d8188720 2023-10-16T08:38:51.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f63dffff640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63d804e6a0 con 0x7f63d80ed5c0 2023-10-16T08:38:51.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f569e7fc640 1 -- 172.21.15.40:0/2993798617 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56a00ade00 msgr2=0x7f56a014c9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f569e7fc640 1 --2- 172.21.15.40:0/2993798617 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56a00ade00 0x7f56a014c9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f569e7fc640 1 -- 172.21.15.40:0/2993798617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a015d320 msgr2=0x7f56a0155ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f569e7fc640 1 --2- 172.21.15.40:0/2993798617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a015d320 0x7f56a0155ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f569e7fc640 1 -- 172.21.15.40:0/2993798617 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56940094d0 con 0x7f56a015cf40 2023-10-16T08:38:51.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fe017ad8640 1 -- 172.21.15.40:0/2157627838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0100c9470 con 0x7fe0100ce7c0 2023-10-16T08:38:51.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fe017ad8640 1 -- 172.21.15.40:0/2157627838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe010158a40 con 0x7fe0100ce7c0 2023-10-16T08:38:51.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f1f299a8640 1 Processor -- start 2023-10-16T08:38:51.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f1f299a8640 1 -- start start 2023-10-16T08:38:51.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fb7a9b04640 1 --2- 172.21.15.40:0/358879601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7a40add60 0x7fb7a4089b40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb79400b410 tx=0x7fb79400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:51.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fe0067fc640 1 -- 172.21.15.40:0/2157627838 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0000129b0 con 0x7fe0100ce7c0 2023-10-16T08:38:51.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f569e7fc640 1 --2- 172.21.15.40:0/2993798617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56a015cf40 0x7f56a01601a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f569800bd70 tx=0x7f569800be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:51.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f63deffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63d8188720 0x7f63d804e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f63df7fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63d80ed5c0 0x7f63d80eda40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f63df7fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63d80ed5c0 0x7f63d80eda40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.40:54666/0 (socket says 172.21.15.40:54666) 2023-10-16T08:38:51.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f63df7fe640 1 -- 172.21.15.40:0/3579656516 learned_addr learned my addr 172.21.15.40:0/3579656516 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:51.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f9a61588640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9a5c156a20 0x7f9a5c153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fb799ffb640 1 -- 172.21.15.40:0/358879601 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794011830 con 0x7fb7a40add60 2023-10-16T08:38:51.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f9a60d87640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c0adb80 0x7f9a5c14cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f9a60d87640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c0adb80 0x7f9a5c14cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.40:54004/0 (socket says 172.21.15.40:54004) 2023-10-16T08:38:51.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f9a60d87640 1 -- 172.21.15.40:0/449083186 learned_addr learned my addr 172.21.15.40:0/449083186 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:51.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f51315fa640 1 -- 172.21.15.40:0/795896490 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f512c0a8c40 con 0x7f512c0b05d0 2023-10-16T08:38:51.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f51315fa640 1 -- 172.21.15.40:0/795896490 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f512c0ab040 con 0x7f512c0b05d0 2023-10-16T08:38:51.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f51315fa640 1 -- 172.21.15.40:0/795896490 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f512c067570 con 0x7f512c0b05d0 2023-10-16T08:38:51.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fb7ab567640 1 -- 172.21.15.40:0/358879601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a42139d0 con 0x7fb7a40add60 2023-10-16T08:38:51.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fb7ab567640 1 -- 172.21.15.40:0/358879601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a4213f90 con 0x7fb7a40add60 2023-10-16T08:38:51.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f993376e640 1 -- 172.21.15.40:0/1377437207 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f992c0b3f10 con 0x7f992c15d3f0 2023-10-16T08:38:51.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fb799ffb640 1 -- 172.21.15.40:0/358879601 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb794011990 con 0x7fb7a40add60 2023-10-16T08:38:51.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fb799ffb640 1 -- 172.21.15.40:0/358879601 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb794011d20 con 0x7fb7a40add60 2023-10-16T08:38:51.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fe0067fc640 1 -- 172.21.15.40:0/2157627838 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe000012d60 con 0x7fe0100ce7c0 2023-10-16T08:38:51.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7fe017ad8640 1 -- 172.21.15.40:0/2157627838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdff8005260 con 0x7fe0100ce7c0 2023-10-16T08:38:51.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f63df7fe640 1 -- 172.21.15.40:0/3579656516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63d8188720 msgr2=0x7f63d804e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f63df7fe640 1 --2- 172.21.15.40:0/3579656516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63d8188720 0x7f63d804e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f63df7fe640 1 -- 172.21.15.40:0/3579656516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d80de190 msgr2=0x7f63d80de610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f63df7fe640 1 --2- 172.21.15.40:0/3579656516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d80de190 0x7f63d80de610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f63df7fe640 1 -- 172.21.15.40:0/3579656516 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d804d0c0 con 0x7f63d80ed5c0 2023-10-16T08:38:51.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f1f299a8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2406d590 0x7f1f240cc680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f1f299a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f2415b6e0 0x7f1f240b3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f1f299a8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f24162c10 0x7f1f24156410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f1f299a8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f240cc260 con 0x7f1f2406d590 2023-10-16T08:38:51.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f1f299a8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f240cd9f0 con 0x7f1f2415b6e0 2023-10-16T08:38:51.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f1f299a8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f240cbba0 con 0x7f1f24162c10 2023-10-16T08:38:51.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f56877fe640 1 -- 172.21.15.40:0/2993798617 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5698014440 con 0x7f56a015cf40 2023-10-16T08:38:51.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f56877fe640 1 -- 172.21.15.40:0/2993798617 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56980145c0 con 0x7f56a015cf40 2023-10-16T08:38:51.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f56877fe640 1 -- 172.21.15.40:0/2993798617 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5698014970 con 0x7f56a015cf40 2023-10-16T08:38:51.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f56a5365640 1 -- 172.21.15.40:0/2993798617 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56a015e650 con 0x7f56a015cf40 2023-10-16T08:38:51.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f56a5365640 1 -- 172.21.15.40:0/2993798617 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56a015ec10 con 0x7f56a015cf40 2023-10-16T08:38:51.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f9a5bfff640 1 --2- 172.21.15.40:0/449083186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a5c0cb370 0x7f9a5c14f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f63df7fe640 1 --2- 172.21.15.40:0/3579656516 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63d80ed5c0 0x7f63d80eda40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f63e0072c80 tx=0x7f63c8005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:51.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f1f22ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2406d590 0x7f1f240cc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f1f22ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2406d590 0x7f1f240cc680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58394/0 (socket says 172.21.15.40:58394) 2023-10-16T08:38:51.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f1f22ffd640 1 -- 172.21.15.40:0/3240932309 learned_addr learned my addr 172.21.15.40:0/3240932309 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:51.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f1f22ffd640 1 -- 172.21.15.40:0/3240932309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f24162c10 msgr2=0x7f1f24156410 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-16T08:38:51.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f1f22ffd640 1 --2- 172.21.15.40:0/3240932309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f24162c10 0x7f1f24156410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f1f22ffd640 1 -- 172.21.15.40:0/3240932309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f2415b6e0 msgr2=0x7f1f240b3570 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-16T08:38:51.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f1f22ffd640 1 --2- 172.21.15.40:0/3240932309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f2415b6e0 0x7f1f240b3570 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f1f22ffd640 1 -- 172.21.15.40:0/3240932309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f14009420 con 0x7f1f2406d590 2023-10-16T08:38:51.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.956+0000 7f9921ffb640 1 -- 172.21.15.40:0/1377437207 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f992400d9a0 con 0x7f992c15d3f0 2023-10-16T08:38:51.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f9921ffb640 1 --2- 172.21.15.40:0/1377437207 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f99100664a0 0x7f9910068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f9921ffb640 1 -- 172.21.15.40:0/1377437207 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9924086500 con 0x7f992c15d3f0 2023-10-16T08:38:51.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f9a60d87640 1 -- 172.21.15.40:0/449083186 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9a5c156a20 msgr2=0x7f9a5c153180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f9a60d87640 1 --2- 172.21.15.40:0/449083186 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9a5c156a20 0x7f9a5c153180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f9a60d87640 1 -- 172.21.15.40:0/449083186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a5c0cb370 msgr2=0x7f9a5c14f200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f9a60d87640 1 --2- 172.21.15.40:0/449083186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a5c0cb370 0x7f9a5c14f200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f9a60d87640 1 -- 172.21.15.40:0/449083186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a5400d050 con 0x7f9a5c0adb80 2023-10-16T08:38:51.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7fb7ab567640 1 -- 172.21.15.40:0/358879601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb784005260 con 0x7fb7a40add60 2023-10-16T08:38:51.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f9a60d87640 1 --2- 172.21.15.40:0/449083186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c0adb80 0x7f9a5c14cc30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00c490 tx=0x7f9a4c001e80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:51.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f1f22ffd640 1 --2- 172.21.15.40:0/3240932309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2406d590 0x7f1f240cc680 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c007ef0 tx=0x7f1f1c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:51.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f51137fe640 1 -- 172.21.15.40:0/795896490 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5120009b80 con 0x7f512c0b05d0 2023-10-16T08:38:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f993150a640 1 --2- 172.21.15.40:0/1377437207 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f99100664a0 0x7f9910068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f51137fe640 1 --2- 172.21.15.40:0/795896490 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f5118066570 0x7f5118068a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f51137fe640 1 -- 172.21.15.40:0/795896490 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f51200849f0 con 0x7f512c0b05d0 2023-10-16T08:38:51.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7fe0067fc640 1 -- 172.21.15.40:0/2157627838 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe000007020 con 0x7fe0100ce7c0 2023-10-16T08:38:51.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f56a5365640 1 -- 172.21.15.40:0/2993798617 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5680005260 con 0x7f56a015cf40 2023-10-16T08:38:51.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f512affd640 1 --2- 172.21.15.40:0/795896490 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f5118066570 0x7f5118068a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7fe0067fc640 1 --2- 172.21.15.40:0/2157627838 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdff40664d0 0x7fdff4068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7fe0067fc640 1 -- 172.21.15.40:0/2157627838 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe000084690 con 0x7fe0100ce7c0 2023-10-16T08:38:51.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f993150a640 1 --2- 172.21.15.40:0/1377437207 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f99100664a0 0x7f9910068960 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f991c009980 tx=0x7f991c0090e0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:51.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7fb799ffb640 1 -- 172.21.15.40:0/358879601 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb79401a400 con 0x7fb7a40add60 2023-10-16T08:38:51.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7fb799ffb640 1 --2- 172.21.15.40:0/358879601 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb790066680 0x7fb790068b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7fb799ffb640 1 -- 172.21.15.40:0/358879601 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb794083dc0 con 0x7fb7a40add60 2023-10-16T08:38:51.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7fe015874640 1 --2- 172.21.15.40:0/2157627838 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdff40664d0 0x7fdff4068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7fb7a9303640 1 --2- 172.21.15.40:0/358879601 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb790066680 0x7fb790068b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7f512affd640 1 --2- 172.21.15.40:0/795896490 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f5118066570 0x7f5118068a30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f51240113e0 tx=0x7f512400b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:51.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7fb7a9303640 1 --2- 172.21.15.40:0/358879601 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb790066680 0x7fb790068b40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00ae000 tx=0x7fb7a00b5040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:51.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.960+0000 7fe015874640 1 --2- 172.21.15.40:0/2157627838 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdff40664d0 0x7fdff4068990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe00c0ad000 tx=0x7fe00c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:51.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.964+0000 7f51137fe640 1 -- 172.21.15.40:0/795896490 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f5120051ee0 con 0x7f512c0b05d0 2023-10-16T08:38:51.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.964+0000 7f56877fe640 1 -- 172.21.15.40:0/2993798617 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f569801e440 con 0x7f56a015cf40 2023-10-16T08:38:51.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.964+0000 7f1f0bfff640 1 -- 172.21.15.40:0/3240932309 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c014620 con 0x7f1f2406d590 2023-10-16T08:38:51.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.964+0000 7f56877fe640 1 --2- 172.21.15.40:0/2993798617 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f5688066600 0x7f5688068ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.964+0000 7f569effd640 1 --2- 172.21.15.40:0/2993798617 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f5688066600 0x7f5688068ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.968+0000 7f1f299a8640 1 -- 172.21.15.40:0/3240932309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f240cbe80 con 0x7f1f2406d590 2023-10-16T08:38:51.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.968+0000 7f1f299a8640 1 -- 172.21.15.40:0/3240932309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f240ccbc0 con 0x7f1f2406d590 2023-10-16T08:38:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.968+0000 7f569effd640 1 --2- 172.21.15.40:0/2993798617 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f5688066600 0x7f5688068ac0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f56a01566e0 tx=0x7f5690006ca0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:51.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.968+0000 7f1f09ffb640 1 -- 172.21.15.40:0/3240932309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f04005260 con 0x7f1f2406d590 2023-10-16T08:38:51.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.968+0000 7f56877fe640 1 -- 172.21.15.40:0/2993798617 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5698055920 con 0x7f56a015cf40 2023-10-16T08:38:51.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.968+0000 7fb799ffb640 1 -- 172.21.15.40:0/358879601 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fb794051220 con 0x7fb7a40add60 2023-10-16T08:38:51.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.968+0000 7f1f0bfff640 1 -- 172.21.15.40:0/3240932309 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f1c009ac0 con 0x7f1f2406d590 2023-10-16T08:38:51.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.968+0000 7f1f0bfff640 1 -- 172.21.15.40:0/3240932309 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f1c014980 con 0x7f1f2406d590 2023-10-16T08:38:51.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.972+0000 7f63c77fe640 1 -- 172.21.15.40:0/3579656516 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e0133070 con 0x7f63d80ed5c0 2023-10-16T08:38:51.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.972+0000 7f63c77fe640 1 -- 172.21.15.40:0/3579656516 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e0069e90 con 0x7f63d80ed5c0 2023-10-16T08:38:51.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.972+0000 7f63c77fe640 1 -- 172.21.15.40:0/3579656516 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e0068670 con 0x7f63d80ed5c0 2023-10-16T08:38:51.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.972+0000 7f63dffff640 1 -- 172.21.15.40:0/3579656516 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63d804d3a0 con 0x7f63d80ed5c0 2023-10-16T08:38:51.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.972+0000 7f63dffff640 1 -- 172.21.15.40:0/3579656516 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63d804d960 con 0x7f63d80ed5c0 2023-10-16T08:38:51.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.972+0000 7fe0067fc640 1 -- 172.21.15.40:0/2157627838 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fe000051af0 con 0x7fe0100ce7c0 2023-10-16T08:38:51.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.972+0000 7f9921ffb640 1 -- 172.21.15.40:0/1377437207 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f9924053a20 con 0x7f992c15d3f0 2023-10-16T08:38:51.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.972+0000 7f9a597fa640 1 -- 172.21.15.40:0/449083186 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c003600 con 0x7f9a5c0adb80 2023-10-16T08:38:51.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.972+0000 7f9a62feb640 1 -- 172.21.15.40:0/449083186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a5c1564d0 con 0x7f9a5c0adb80 2023-10-16T08:38:51.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.972+0000 7f9a62feb640 1 -- 172.21.15.40:0/449083186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a5c14e990 con 0x7f9a5c0adb80 2023-10-16T08:38:51.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.972+0000 7f9a597fa640 1 -- 172.21.15.40:0/449083186 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a4c003780 con 0x7f9a5c0adb80 2023-10-16T08:38:51.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.972+0000 7f9a597fa640 1 -- 172.21.15.40:0/449083186 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4c003b70 con 0x7f9a5c0adb80 2023-10-16T08:38:51.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.972+0000 7f9a62feb640 1 -- 172.21.15.40:0/449083186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a3c005260 con 0x7f9a5c0adb80 2023-10-16T08:38:51.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.972+0000 7f63dffff640 1 -- 172.21.15.40:0/3579656516 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c0005260 con 0x7f63d80ed5c0 2023-10-16T08:38:51.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.976+0000 7f9a597fa640 1 -- 172.21.15.40:0/449083186 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9a4c003cd0 con 0x7f9a5c0adb80 2023-10-16T08:38:51.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.976+0000 7f9a597fa640 1 --2- 172.21.15.40:0/449083186 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f9a480664a0 0x7f9a48068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.976+0000 7f63c77fe640 1 -- 172.21.15.40:0/3579656516 <== mon.2 v2:172.21.15.86:3301/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f63e01257d0 con 0x7f63d80ed5c0 2023-10-16T08:38:51.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.976+0000 7f9a597fa640 1 -- 172.21.15.40:0/449083186 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a4c083900 con 0x7f9a5c0adb80 2023-10-16T08:38:51.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.976+0000 7f9a5bfff640 1 --2- 172.21.15.40:0/449083186 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f9a480664a0 0x7f9a48068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.976+0000 7f63c77fe640 1 --2- 172.21.15.40:0/3579656516 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f63cc0664f0 0x7f63cc0689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.976+0000 7f63c77fe640 1 -- 172.21.15.40:0/3579656516 <== mon.2 v2:172.21.15.86:3301/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63e0197860 con 0x7f63d80ed5c0 2023-10-16T08:38:51.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.976+0000 7f63deffd640 1 --2- 172.21.15.40:0/3579656516 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f63cc0664f0 0x7f63cc0689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.976+0000 7f9a5bfff640 1 --2- 172.21.15.40:0/449083186 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f9a480664a0 0x7f9a48068960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9a50006fd0 tx=0x7f9a50008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:51.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.976+0000 7f63deffd640 1 --2- 172.21.15.40:0/3579656516 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f63cc0664f0 0x7f63cc0689b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f63d0001000 tx=0x7f63d0001090 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:51.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.980+0000 7f56877fe640 1 -- 172.21.15.40:0/2993798617 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f5698051c60 con 0x7f56a015cf40 2023-10-16T08:38:51.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.980+0000 7f1f0bfff640 1 -- 172.21.15.40:0/3240932309 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1f1c014ae0 con 0x7f1f2406d590 2023-10-16T08:38:51.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.980+0000 7f1f0bfff640 1 --2- 172.21.15.40:0/3240932309 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1f100664d0 0x7f1f10068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:51.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.980+0000 7f1f0bfff640 1 -- 172.21.15.40:0/3240932309 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f1c085680 con 0x7f1f2406d590 2023-10-16T08:38:51.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.980+0000 7f1f227fc640 1 --2- 172.21.15.40:0/3240932309 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1f100664d0 0x7f1f10068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:51.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.980+0000 7f1f227fc640 1 --2- 172.21.15.40:0/3240932309 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1f100664d0 0x7f1f10068990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1f240b0600 tx=0x7f1f18007a50 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:51.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.988+0000 7f9a597fa640 1 -- 172.21.15.40:0/449083186 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f9a4c050d60 con 0x7f9a5c0adb80 2023-10-16T08:38:51.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:50.988+0000 7f63c77fe640 1 -- 172.21.15.40:0/3579656516 <== mon.2 v2:172.21.15.86:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f63e0164cc0 con 0x7f63d80ed5c0 2023-10-16T08:38:51.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.004+0000 7f1f0bfff640 1 -- 172.21.15.40:0/3240932309 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f1f1c052b70 con 0x7f1f2406d590 2023-10-16T08:38:51.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.280+0000 7fb7ab567640 1 -- 172.21.15.40:0/358879601 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb784005510 con 0x7fb7a40add60 2023-10-16T08:38:51.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.280+0000 7fb799ffb640 1 -- 172.21.15.40:0/358879601 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb794015020 con 0x7fb7a40add60 2023-10-16T08:38:51.284 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-16T08:38:51.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.280+0000 7f51315fa640 1 -- 172.21.15.40:0/795896490 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f512c0cdc50 con 0x7f512c0b05d0 2023-10-16T08:38:51.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.280+0000 7f51137fe640 1 -- 172.21.15.40:0/795896490 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5120055500 con 0x7f512c0b05d0 2023-10-16T08:38:51.286 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2023-10-16T08:38:51.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7fb7ab567640 1 -- 172.21.15.40:0/358879601 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb790066680 msgr2=0x7fb790068b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7fb7ab567640 1 --2- 172.21.15.40:0/358879601 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb790066680 0x7fb790068b40 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00ae000 tx=0x7fb7a00b5040 comp rx=0 tx=0).stop 2023-10-16T08:38:51.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7fb7ab567640 1 -- 172.21.15.40:0/358879601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7a40add60 msgr2=0x7fb7a4089b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7fb7ab567640 1 --2- 172.21.15.40:0/358879601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7a40add60 0x7fb7a4089b40 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb79400b410 tx=0x7fb79400b8d0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7fb7ab567640 1 -- 172.21.15.40:0/358879601 shutdown_connections 2023-10-16T08:38:51.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7fb7ab567640 1 --2- 172.21.15.40:0/358879601 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb790066680 0x7fb790068b40 secure :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb7a00ae000 tx=0x7fb7a00b5040 comp rx=0 tx=0).stop 2023-10-16T08:38:51.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7fb7ab567640 1 --2- 172.21.15.40:0/358879601 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb7a40add60 0x7fb7a4089b40 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7fb7ab567640 1 --2- 172.21.15.40:0/358879601 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7a40ad980 0x7fb7a40c87f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7fb7ab567640 1 --2- 172.21.15.40:0/358879601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7a406c650 0x7fb7a42130c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7fb7ab567640 1 -- 172.21.15.40:0/358879601 >> 172.21.15.40:0/358879601 conn(0x7fb7a40b0470 msgr2=0x7fb7a40ae2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:51.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7fb7ab567640 1 -- 172.21.15.40:0/358879601 shutdown_connections 2023-10-16T08:38:51.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7fb7ab567640 1 -- 172.21.15.40:0/358879601 wait complete. 2023-10-16T08:38:51.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7f51315fa640 1 -- 172.21.15.40:0/795896490 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f5118066570 msgr2=0x7f5118068a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7f51315fa640 1 --2- 172.21.15.40:0/795896490 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f5118066570 0x7f5118068a30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f51240113e0 tx=0x7f512400b040 comp rx=0 tx=0).stop 2023-10-16T08:38:51.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7f51315fa640 1 -- 172.21.15.40:0/795896490 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f512c0b05d0 msgr2=0x7f512c0a84d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7f51315fa640 1 --2- 172.21.15.40:0/795896490 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f512c0b05d0 0x7f512c0a84d0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f512000d830 tx=0x7f512000dcf0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7f51315fa640 1 -- 172.21.15.40:0/795896490 shutdown_connections 2023-10-16T08:38:51.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7f51315fa640 1 --2- 172.21.15.40:0/795896490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f512c158cd0 0x7f512c205d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7f51315fa640 1 --2- 172.21.15.40:0/795896490 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f5118066570 0x7f5118068a30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7f51315fa640 1 --2- 172.21.15.40:0/795896490 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f512c0b05d0 0x7f512c0a84d0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7f51315fa640 1 --2- 172.21.15.40:0/795896490 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f512c06c8a0 0x7f512c0c2890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7f51315fa640 1 -- 172.21.15.40:0/795896490 >> 172.21.15.40:0/795896490 conn(0x7f512c0cb090 msgr2=0x7f512c0ae070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:51.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7f51315fa640 1 -- 172.21.15.40:0/795896490 shutdown_connections 2023-10-16T08:38:51.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.284+0000 7f51315fa640 1 -- 172.21.15.40:0/795896490 wait complete. 2023-10-16T08:38:51.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.292+0000 7fe017ad8640 1 -- 172.21.15.40:0/2157627838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fdff80057e0 con 0x7fe0100ce7c0 2023-10-16T08:38:51.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.292+0000 7fe0067fc640 1 -- 172.21.15.40:0/2157627838 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe000055110 con 0x7fe0100ce7c0 2023-10-16T08:38:51.298 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2023-10-16T08:38:51.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.296+0000 7f993376e640 1 -- 172.21.15.40:0/1377437207 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f992c15b120 con 0x7f992c15d3f0 2023-10-16T08:38:51.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.296+0000 7f9921ffb640 1 -- 172.21.15.40:0/1377437207 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f992c15b120 con 0x7f992c15d3f0 2023-10-16T08:38:51.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.296+0000 7fe017ad8640 1 -- 172.21.15.40:0/2157627838 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdff40664d0 msgr2=0x7fdff4068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.302 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2023-10-16T08:38:51.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.296+0000 7fe017ad8640 1 --2- 172.21.15.40:0/2157627838 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdff40664d0 0x7fdff4068990 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe00c0ad000 tx=0x7fe00c008040 comp rx=0 tx=0).stop 2023-10-16T08:38:51.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.296+0000 7fe017ad8640 1 -- 172.21.15.40:0/2157627838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0100ce7c0 msgr2=0x7fe01015b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.296+0000 7fe017ad8640 1 --2- 172.21.15.40:0/2157627838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0100ce7c0 0x7fe01015b7c0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe00000c450 tx=0x7fe00000c910 comp rx=0 tx=0).stop 2023-10-16T08:38:51.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.300+0000 7fe017ad8640 1 -- 172.21.15.40:0/2157627838 shutdown_connections 2023-10-16T08:38:51.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.300+0000 7fe017ad8640 1 --2- 172.21.15.40:0/2157627838 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdff40664d0 0x7fdff4068990 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.300+0000 7fe017ad8640 1 --2- 172.21.15.40:0/2157627838 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe0101591e0 0x7fe01015bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.300+0000 7fe017ad8640 1 --2- 172.21.15.40:0/2157627838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0100ce7c0 0x7fe01015b7c0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.300+0000 7fe017ad8640 1 --2- 172.21.15.40:0/2157627838 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe01006fca0 0x7fe010158500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.300+0000 7fe017ad8640 1 -- 172.21.15.40:0/2157627838 >> 172.21.15.40:0/2157627838 conn(0x7fe0100b0be0 msgr2=0x7fe010163630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:51.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.300+0000 7fe017ad8640 1 -- 172.21.15.40:0/2157627838 shutdown_connections 2023-10-16T08:38:51.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.300+0000 7fe017ad8640 1 -- 172.21.15.40:0/2157627838 wait complete. 2023-10-16T08:38:51.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.300+0000 7f9a62feb640 1 -- 172.21.15.40:0/449083186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f9a3c0057e0 con 0x7f9a5c0adb80 2023-10-16T08:38:51.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.300+0000 7f56a5365640 1 -- 172.21.15.40:0/2993798617 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f56800057e0 con 0x7f56a015cf40 2023-10-16T08:38:51.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.300+0000 7f56877fe640 1 -- 172.21.15.40:0/2993798617 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5698019070 con 0x7f56a015cf40 2023-10-16T08:38:51.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f993376e640 1 -- 172.21.15.40:0/1377437207 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f99100664a0 msgr2=0x7f9910068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f993376e640 1 --2- 172.21.15.40:0/1377437207 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f99100664a0 0x7f9910068960 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f991c009980 tx=0x7f991c0090e0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f993376e640 1 -- 172.21.15.40:0/1377437207 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f992c15d3f0 msgr2=0x7f992c1e6e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f993376e640 1 --2- 172.21.15.40:0/1377437207 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f992c15d3f0 0x7f992c1e6e40 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f992400f780 tx=0x7f992400fc40 comp rx=0 tx=0).stop 2023-10-16T08:38:51.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f993376e640 1 -- 172.21.15.40:0/1377437207 shutdown_connections 2023-10-16T08:38:51.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f993376e640 1 --2- 172.21.15.40:0/1377437207 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f992c15d3f0 0x7f992c1e6e40 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f993376e640 1 --2- 172.21.15.40:0/1377437207 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f992c158cd0 0x7f992c0b39d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f993376e640 1 --2- 172.21.15.40:0/1377437207 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f99100664a0 0x7f9910068960 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f991c009980 tx=0x7f991c0090e0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f993376e640 1 --2- 172.21.15.40:0/1377437207 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f992c0b05d0 0x7f992c0aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f993376e640 1 -- 172.21.15.40:0/1377437207 >> 172.21.15.40:0/1377437207 conn(0x7f992c0cb090 msgr2=0x7f992c157390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:51.311 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2023-10-16T08:38:51.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.300+0000 7f9a597fa640 1 -- 172.21.15.40:0/449083186 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f9a4c017090 con 0x7f9a5c0adb80 2023-10-16T08:38:51.312 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2023-10-16T08:38:51.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f63dffff640 1 -- 172.21.15.40:0/3579656516 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f63c00057e0 con 0x7f63d80ed5c0 2023-10-16T08:38:51.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f56a5365640 1 -- 172.21.15.40:0/2993798617 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f5688066600 msgr2=0x7f5688068ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f56a5365640 1 --2- 172.21.15.40:0/2993798617 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f5688066600 0x7f5688068ac0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f56a01566e0 tx=0x7f5690006ca0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f56a5365640 1 -- 172.21.15.40:0/2993798617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56a015cf40 msgr2=0x7f56a01601a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f56a5365640 1 --2- 172.21.15.40:0/2993798617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56a015cf40 0x7f56a01601a0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f569800bd70 tx=0x7f569800be60 comp rx=0 tx=0).stop 2023-10-16T08:38:51.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f993376e640 1 -- 172.21.15.40:0/1377437207 shutdown_connections 2023-10-16T08:38:51.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f56a5365640 1 -- 172.21.15.40:0/2993798617 shutdown_connections 2023-10-16T08:38:51.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f56a5365640 1 --2- 172.21.15.40:0/2993798617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56a015d320 0x7f56a0155ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f56a5365640 1 --2- 172.21.15.40:0/2993798617 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f5688066600 0x7f5688068ac0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f993376e640 1 -- 172.21.15.40:0/1377437207 wait complete. 2023-10-16T08:38:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f56a5365640 1 --2- 172.21.15.40:0/2993798617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f56a015cf40 0x7f56a01601a0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f56a5365640 1 --2- 172.21.15.40:0/2993798617 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f56a00ade00 0x7f56a014c9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f56a5365640 1 -- 172.21.15.40:0/2993798617 >> 172.21.15.40:0/2993798617 conn(0x7f56a00ce960 msgr2=0x7f56a00ab190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:51.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f9a42ffd640 1 -- 172.21.15.40:0/449083186 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f9a480664a0 msgr2=0x7f9a48068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f56a5365640 1 -- 172.21.15.40:0/2993798617 shutdown_connections 2023-10-16T08:38:51.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f9a42ffd640 1 --2- 172.21.15.40:0/449083186 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f9a480664a0 0x7f9a48068960 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f9a50006fd0 tx=0x7f9a50008040 comp rx=0 tx=0).stop 2023-10-16T08:38:51.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f56a5365640 1 -- 172.21.15.40:0/2993798617 wait complete. 2023-10-16T08:38:51.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f9a42ffd640 1 -- 172.21.15.40:0/449083186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c0adb80 msgr2=0x7f9a5c14cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f9a42ffd640 1 --2- 172.21.15.40:0/449083186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c0adb80 0x7f9a5c14cc30 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f9a4c00c490 tx=0x7f9a4c001e80 comp rx=0 tx=0).stop 2023-10-16T08:38:51.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.304+0000 7f63c77fe640 1 -- 172.21.15.40:0/3579656516 <== mon.2 v2:172.21.15.86:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f63e01682e0 con 0x7f63d80ed5c0 2023-10-16T08:38:51.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.308+0000 7f9a42ffd640 1 -- 172.21.15.40:0/449083186 shutdown_connections 2023-10-16T08:38:51.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.308+0000 7f9a42ffd640 1 --2- 172.21.15.40:0/449083186 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9a5c156a20 0x7f9a5c153180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.320 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-16T08:38:51.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.308+0000 7f9a42ffd640 1 --2- 172.21.15.40:0/449083186 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f9a480664a0 0x7f9a48068960 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.308+0000 7f9a42ffd640 1 --2- 172.21.15.40:0/449083186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a5c0cb370 0x7f9a5c14f200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.308+0000 7f9a42ffd640 1 --2- 172.21.15.40:0/449083186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c0adb80 0x7f9a5c14cc30 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.308+0000 7f9a42ffd640 1 -- 172.21.15.40:0/449083186 >> 172.21.15.40:0/449083186 conn(0x7f9a5c0b0470 msgr2=0x7f9a5c157070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:51.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.308+0000 7f9a42ffd640 1 -- 172.21.15.40:0/449083186 shutdown_connections 2023-10-16T08:38:51.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.308+0000 7f9a42ffd640 1 -- 172.21.15.40:0/449083186 wait complete. 2023-10-16T08:38:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.308+0000 7f63dffff640 1 -- 172.21.15.40:0/3579656516 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f63cc0664f0 msgr2=0x7f63cc0689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.308+0000 7f63dffff640 1 --2- 172.21.15.40:0/3579656516 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f63cc0664f0 0x7f63cc0689b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f63d0001000 tx=0x7f63d0001090 comp rx=0 tx=0).stop 2023-10-16T08:38:51.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.308+0000 7f63dffff640 1 -- 172.21.15.40:0/3579656516 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63d80ed5c0 msgr2=0x7f63d80eda40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.308+0000 7f63dffff640 1 --2- 172.21.15.40:0/3579656516 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63d80ed5c0 0x7f63d80eda40 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f63e0072c80 tx=0x7f63c8005800 comp rx=0 tx=0).stop 2023-10-16T08:38:51.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.308+0000 7f63dffff640 1 -- 172.21.15.40:0/3579656516 shutdown_connections 2023-10-16T08:38:51.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.308+0000 7f63dffff640 1 --2- 172.21.15.40:0/3579656516 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63d80ed5c0 0x7f63d80eda40 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.308+0000 7f63dffff640 1 --2- 172.21.15.40:0/3579656516 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f63cc0664f0 0x7f63cc0689b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.308+0000 7f63dffff640 1 --2- 172.21.15.40:0/3579656516 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63d80de190 0x7f63d80de610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.308+0000 7f63dffff640 1 --2- 172.21.15.40:0/3579656516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f63d8188720 0x7f63d804e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.308+0000 7f63dffff640 1 -- 172.21.15.40:0/3579656516 >> 172.21.15.40:0/3579656516 conn(0x7f63d8054bc0 msgr2=0x7f63d804f210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:51.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.308+0000 7f63dffff640 1 -- 172.21.15.40:0/3579656516 shutdown_connections 2023-10-16T08:38:51.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.308+0000 7f63dffff640 1 -- 172.21.15.40:0/3579656516 wait complete. 2023-10-16T08:38:51.326 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.3 2023-10-16T08:38:51.327 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.1 2023-10-16T08:38:51.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.324+0000 7f1f09ffb640 1 -- 172.21.15.40:0/3240932309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f1f040057e0 con 0x7f1f2406d590 2023-10-16T08:38:51.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.324+0000 7f1f0bfff640 1 -- 172.21.15.40:0/3240932309 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1f1c056190 con 0x7f1f2406d590 2023-10-16T08:38:51.329 INFO:teuthology.orchestra.run.smithi040.stdout:55834574853 2023-10-16T08:38:51.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.328+0000 7f1f09ffb640 1 -- 172.21.15.40:0/3240932309 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1f100664d0 msgr2=0x7f1f10068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.328+0000 7f1f09ffb640 1 --2- 172.21.15.40:0/3240932309 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1f100664d0 0x7f1f10068990 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1f240b0600 tx=0x7f1f18007a50 comp rx=0 tx=0).stop 2023-10-16T08:38:51.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.328+0000 7f1f09ffb640 1 -- 172.21.15.40:0/3240932309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2406d590 msgr2=0x7f1f240cc680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:51.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.328+0000 7f1f09ffb640 1 --2- 172.21.15.40:0/3240932309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2406d590 0x7f1f240cc680 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1f1c007ef0 tx=0x7f1f1c00e4b0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.328+0000 7f1f09ffb640 1 -- 172.21.15.40:0/3240932309 shutdown_connections 2023-10-16T08:38:51.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.328+0000 7f1f09ffb640 1 --2- 172.21.15.40:0/3240932309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1f24162c10 0x7f1f24156410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.328+0000 7f1f09ffb640 1 --2- 172.21.15.40:0/3240932309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f2415b6e0 0x7f1f240b3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.328+0000 7f1f09ffb640 1 --2- 172.21.15.40:0/3240932309 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1f100664d0 0x7f1f10068990 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.328+0000 7f1f09ffb640 1 --2- 172.21.15.40:0/3240932309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1f2406d590 0x7f1f240cc680 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:51.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.328+0000 7f1f09ffb640 1 -- 172.21.15.40:0/3240932309 >> 172.21.15.40:0/3240932309 conn(0x7f1f240661e0 msgr2=0x7f1f240ce120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:51.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.328+0000 7f1f09ffb640 1 -- 172.21.15.40:0/3240932309 shutdown_connections 2023-10-16T08:38:51.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:51.328+0000 7f1f09ffb640 1 -- 172.21.15.40:0/3240932309 wait complete. 2023-10-16T08:38:51.336 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.2 2023-10-16T08:38:51.338 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-10-16T08:38:51.341 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2023-10-16T08:38:51.343 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.7 2023-10-16T08:38:51.344 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574855 for osd.5 2023-10-16T08:38:51.352 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.6 2023-10-16T08:38:52.327 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-16T08:38:52.327 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-16T08:38:52.338 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-16T08:38:52.339 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-16T08:38:52.342 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-16T08:38:52.344 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-16T08:38:52.344 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-16T08:38:52.353 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-16T08:38:52.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.496+0000 7f7a8c7b7640 1 Processor -- start 2023-10-16T08:38:52.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.496+0000 7f7a8c7b7640 1 -- start start 2023-10-16T08:38:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.500+0000 7f7a8c7b7640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f7a84158cd0 0x7f7a8415d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.500+0000 7f7a8c7b7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a840b07a0 0x7f7a8406fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.500+0000 7f7a8c7b7640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a8406f270 con 0x7f7a8406c8a0 2023-10-16T08:38:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.500+0000 7f7a8c7b7640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a8406ad40 con 0x7f7a84158cd0 2023-10-16T08:38:52.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.500+0000 7f7a8c7b7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8406b020 con 0x7f7a840b07a0 2023-10-16T08:38:52.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.500+0000 7f7a89d52640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f7a8406c8a0 0x7f7a840b0090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58396/0 (socket says 172.21.15.40:58396) 2023-10-16T08:38:52.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.500+0000 7f7a89d52640 1 -- 172.21.15.40:0/1835118286 learned_addr learned my addr 172.21.15.40:0/1835118286 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:52.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.500+0000 7f7a8a553640 1 --2- 172.21.15.40:0/1835118286 >> v2:172.21.15.86:3300/0 conn(0x7f7a84158cd0 0x7f7a8415d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.500+0000 7f7a8ad54640 1 --2- 172.21.15.40:0/1835118286 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a840b07a0 0x7f7a8406fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.500+0000 7f7a89551640 1 -- 172.21.15.40:0/1835118286 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 160131288 0 0) 0x7f7a8406f270 con 0x7f7a8406c8a0 2023-10-16T08:38:52.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.500+0000 7f7a89551640 1 -- 172.21.15.40:0/1835118286 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a70003530 con 0x7f7a8406c8a0 2023-10-16T08:38:52.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.500+0000 7f7a89551640 1 -- 172.21.15.40:0/1835118286 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3274321352 0 0) 0x7f7a70003530 con 0x7f7a8406c8a0 2023-10-16T08:38:52.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.500+0000 7f7a8a553640 1 -- 172.21.15.40:0/1835118286 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a840b07a0 msgr2=0x7f7a8406fa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:52.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.500+0000 7f7a8a553640 1 --2- 172.21.15.40:0/1835118286 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a840b07a0 0x7f7a8406fa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.500+0000 7f7a8a553640 1 -- 172.21.15.40:0/1835118286 >> v1:172.21.15.40:6789/0 conn(0x7f7a8406c8a0 legacy=0x7f7a840b0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.500+0000 7f7a8a553640 1 -- 172.21.15.40:0/1835118286 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a8406f510 con 0x7f7a84158cd0 2023-10-16T08:38:52.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.504+0000 7f7a8a553640 1 --2- 172.21.15.40:0/1835118286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a84158cd0 0x7f7a8415d3f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00d1e0 tx=0x7f7a7c01ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df490a9d6b8f1f8d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.504+0000 7f7a89551640 1 -- 172.21.15.40:0/1835118286 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a80001680 con 0x7f7a84158cd0 2023-10-16T08:38:52.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.504+0000 7f7a89551640 1 -- 172.21.15.40:0/1835118286 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7c0ab070 con 0x7f7a84158cd0 2023-10-16T08:38:52.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.504+0000 7f7a89551640 1 -- 172.21.15.40:0/1835118286 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7c0a7aa0 con 0x7f7a84158cd0 2023-10-16T08:38:52.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.504+0000 7f7a8c7b7640 1 -- 172.21.15.40:0/1835118286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a84158cd0 msgr2=0x7f7a8415d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.504+0000 7f7a8c7b7640 1 --2- 172.21.15.40:0/1835118286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a84158cd0 0x7f7a8415d3f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c00d1e0 tx=0x7f7a7c01ec50 comp rx=0 tx=0).stop 2023-10-16T08:38:52.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.504+0000 7f7a8c7b7640 1 -- 172.21.15.40:0/1835118286 shutdown_connections 2023-10-16T08:38:52.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.504+0000 7f7a8c7b7640 1 --2- 172.21.15.40:0/1835118286 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a840b07a0 0x7f7a8406fa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.504+0000 7f7a8c7b7640 1 --2- 172.21.15.40:0/1835118286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a84158cd0 0x7f7a8415d3f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.504+0000 7f7a8c7b7640 1 -- 172.21.15.40:0/1835118286 >> 172.21.15.40:0/1835118286 conn(0x7f7a840cb090 msgr2=0x7f7a840cb4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:52.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.504+0000 7f7a8c7b7640 1 -- 172.21.15.40:0/1835118286 shutdown_connections 2023-10-16T08:38:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.504+0000 7f7a8c7b7640 1 -- 172.21.15.40:0/1835118286 wait complete. 2023-10-16T08:38:52.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.508+0000 7f7a8c7b7640 1 Processor -- start 2023-10-16T08:38:52.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.508+0000 7f7a8c7b7640 1 -- start start 2023-10-16T08:38:52.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.508+0000 7f7a8c7b7640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a8406c8a0 0x7f7a840b4030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.508+0000 7f7a8c7b7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a840b07a0 0x7f7a841f9580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.508+0000 7f7a8c7b7640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a84158cd0 0x7f7a840b2650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.508+0000 7f7a8c7b7640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a8414abf0 con 0x7f7a84158cd0 2023-10-16T08:38:52.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.508+0000 7f7a8c7b7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a840b4570 con 0x7f7a840b07a0 2023-10-16T08:38:52.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.508+0000 7f7a8c7b7640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a840b1890 con 0x7f7a8406c8a0 2023-10-16T08:38:52.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.508+0000 7f7a8ad54640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a84158cd0 0x7f7a840b2650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.508+0000 7f7a8ad54640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a84158cd0 0x7f7a840b2650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58400/0 (socket says 172.21.15.40:58400) 2023-10-16T08:38:52.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.508+0000 7f7a8a553640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a8406c8a0 0x7f7a840b4030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.508+0000 7f7a89d52640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a840b07a0 0x7f7a841f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.508+0000 7f7a8ad54640 1 -- 172.21.15.40:0/2583366813 learned_addr learned my addr 172.21.15.40:0/2583366813 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:52.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.512+0000 7f7a8ad54640 1 -- 172.21.15.40:0/2583366813 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a8406c8a0 msgr2=0x7f7a840b4030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.512+0000 7f7a8ad54640 1 --2- 172.21.15.40:0/2583366813 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a8406c8a0 0x7f7a840b4030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.512+0000 7f7a8ad54640 1 -- 172.21.15.40:0/2583366813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a840b07a0 msgr2=0x7f7a841f9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.512+0000 7f7a8ad54640 1 --2- 172.21.15.40:0/2583366813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a840b07a0 0x7f7a841f9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.512+0000 7f7a8ad54640 1 -- 172.21.15.40:0/2583366813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a7c007910 con 0x7f7a84158cd0 2023-10-16T08:38:52.518 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.512+0000 7f7a8ad54640 1 --2- 172.21.15.40:0/2583366813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a84158cd0 0x7f7a840b2650 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400d800 tx=0x7f7a7400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.516+0000 7fcceb1a4640 1 Processor -- start 2023-10-16T08:38:52.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.516+0000 7fcceb1a4640 1 -- start start 2023-10-16T08:38:52.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.520+0000 7fcceb1a4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccdc0e11d0 0x7fccdc03a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.520+0000 7fcceb1a4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fccdc03a820 0x7fccdc009e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.520+0000 7fcceb1a4640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fccdc0e15b0 0x7fccdc00a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.520+0000 7fcceb1a4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccdc054d40 con 0x7fccdc03a820 2023-10-16T08:38:52.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.520+0000 7fcceb1a4640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccdc055020 con 0x7fccdc0e15b0 2023-10-16T08:38:52.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.520+0000 7fcceb1a4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc055300 con 0x7fccdc0e11d0 2023-10-16T08:38:52.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.520+0000 7fcce99a1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fccdc03a820 0x7fccdc009e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.520+0000 7fcce99a1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fccdc03a820 0x7fccdc009e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58402/0 (socket says 172.21.15.40:58402) 2023-10-16T08:38:52.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.520+0000 7fcce99a1640 1 -- 172.21.15.40:0/2664001396 learned_addr learned my addr 172.21.15.40:0/2664001396 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:52.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.520+0000 7fcce99a1640 1 -- 172.21.15.40:0/2664001396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccdc0e11d0 msgr2=0x7fccdc03a2e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T08:38:52.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.520+0000 7fcce99a1640 1 --2- 172.21.15.40:0/2664001396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccdc0e11d0 0x7fccdc03a2e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.520+0000 7fcce99a1640 1 -- 172.21.15.40:0/2664001396 >> v2:172.21.15.86:3300/0 conn(0x7fccdc0e15b0 msgr2=0x7fccdc00a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:52.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.520+0000 7f7a7affd640 1 -- 172.21.15.40:0/2583366813 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7400b6c0 con 0x7f7a84158cd0 2023-10-16T08:38:52.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.520+0000 7fccea9a3640 1 --2- 172.21.15.40:0/2664001396 >> v2:172.21.15.86:3300/0 conn(0x7fccdc0e15b0 0x7fccdc00a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.520+0000 7fcce99a1640 1 --2- 172.21.15.40:0/2664001396 >> v2:172.21.15.86:3300/0 conn(0x7fccdc0e15b0 0x7fccdc00a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.520+0000 7f7a8c7b7640 1 -- 172.21.15.40:0/2583366813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a840b1b70 con 0x7f7a84158cd0 2023-10-16T08:38:52.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.520+0000 7f7a8c7b7640 1 -- 172.21.15.40:0/2583366813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a840b2130 con 0x7f7a84158cd0 2023-10-16T08:38:52.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.520+0000 7fcce99a1640 1 -- 172.21.15.40:0/2664001396 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccdc0555e0 con 0x7fccdc03a820 2023-10-16T08:38:52.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.520+0000 7f7a7affd640 1 -- 172.21.15.40:0/2583366813 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a7400b820 con 0x7f7a84158cd0 2023-10-16T08:38:52.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.520+0000 7f7a7affd640 1 -- 172.21.15.40:0/2583366813 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a7400bbd0 con 0x7f7a84158cd0 2023-10-16T08:38:52.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.524+0000 7f7a7affd640 1 -- 172.21.15.40:0/2583366813 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7a7400bd30 con 0x7f7a84158cd0 2023-10-16T08:38:52.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.524+0000 7fccea9a3640 1 --2- 172.21.15.40:0/2664001396 >> v2:172.21.15.86:3300/0 conn(0x7fccdc0e15b0 0x7fccdc00a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:52.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.524+0000 7f7a8c7b7640 1 -- 172.21.15.40:0/2583366813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6c005260 con 0x7f7a84158cd0 2023-10-16T08:38:52.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.524+0000 7fcce99a1640 1 --2- 172.21.15.40:0/2664001396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc03a820 0x7fccdc009e50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fccdc056d80 tx=0x7fcce001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c259909603084f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.524+0000 7f7a7affd640 1 --2- 172.21.15.40:0/2583366813 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7a680664f0 0x7f7a680689b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.524+0000 7f7a7affd640 1 -- 172.21.15.40:0/2583366813 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7a740839e0 con 0x7f7a84158cd0 2023-10-16T08:38:52.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.524+0000 7f7a8a553640 1 --2- 172.21.15.40:0/2583366813 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7a680664f0 0x7f7a680689b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.524+0000 7f7a8a553640 1 --2- 172.21.15.40:0/2583366813 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7a680664f0 0x7f7a680689b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0b2000 tx=0x7f7a7c00b040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.532+0000 7f7a7affd640 1 -- 172.21.15.40:0/2583366813 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f7a74050f00 con 0x7f7a84158cd0 2023-10-16T08:38:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.532+0000 7fcce91a0640 1 -- 172.21.15.40:0/2664001396 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce00ab600 con 0x7fccdc03a820 2023-10-16T08:38:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.532+0000 7fcce91a0640 1 -- 172.21.15.40:0/2664001396 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce00ab760 con 0x7fccdc03a820 2023-10-16T08:38:52.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.532+0000 7fcceb1a4640 1 -- 172.21.15.40:0/2664001396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc03a820 msgr2=0x7fccdc009e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.532+0000 7fcceb1a4640 1 --2- 172.21.15.40:0/2664001396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc03a820 0x7fccdc009e50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fccdc056d80 tx=0x7fcce001c510 comp rx=0 tx=0).stop 2023-10-16T08:38:52.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.532+0000 7fcceb1a4640 1 -- 172.21.15.40:0/2664001396 shutdown_connections 2023-10-16T08:38:52.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.532+0000 7fcceb1a4640 1 --2- 172.21.15.40:0/2664001396 >> v2:172.21.15.86:3300/0 conn(0x7fccdc0e15b0 0x7fccdc00a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.532+0000 7fcceb1a4640 1 --2- 172.21.15.40:0/2664001396 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc03a820 0x7fccdc009e50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.532+0000 7fcceb1a4640 1 --2- 172.21.15.40:0/2664001396 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccdc0e11d0 0x7fccdc03a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.532+0000 7fcceb1a4640 1 -- 172.21.15.40:0/2664001396 >> 172.21.15.40:0/2664001396 conn(0x7fccdc056dd0 msgr2=0x7fccdc0571e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:52.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.532+0000 7fcceb1a4640 1 -- 172.21.15.40:0/2664001396 shutdown_connections 2023-10-16T08:38:52.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.532+0000 7fcceb1a4640 1 -- 172.21.15.40:0/2664001396 wait complete. 2023-10-16T08:38:52.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.536+0000 7fcceb1a4640 1 Processor -- start 2023-10-16T08:38:52.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.536+0000 7fcceb1a4640 1 -- start start 2023-10-16T08:38:52.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.536+0000 7fcceb1a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccdc03a820 0x7fccdc0db870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.536+0000 7fcceb1a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc0e11d0 0x7fccdc036240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.536+0000 7fcce99a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc0e11d0 0x7fccdc036240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.536+0000 7fcce99a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc0e11d0 0x7fccdc036240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58414/0 (socket says 172.21.15.40:58414) 2023-10-16T08:38:52.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.536+0000 7fccea1a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccdc03a820 0x7fccdc0db870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.536+0000 7fccea1a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccdc03a820 0x7fccdc0db870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.40:54032/0 (socket says 172.21.15.40:54032) 2023-10-16T08:38:52.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.536+0000 7fcceb1a4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccdc0e15b0 0x7fccdc0d9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.536+0000 7fcceb1a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc0db450 con 0x7fccdc0e11d0 2023-10-16T08:38:52.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.536+0000 7fcceb1a4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc0da860 con 0x7fccdc03a820 2023-10-16T08:38:52.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.536+0000 7fcceb1a4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccdc0daab0 con 0x7fccdc0e15b0 2023-10-16T08:38:52.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.536+0000 7fccea1a2640 1 -- 172.21.15.40:0/135104967 learned_addr learned my addr 172.21.15.40:0/135104967 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:52.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7fccea9a3640 1 --2- 172.21.15.40:0/135104967 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccdc0e15b0 0x7fccdc0d9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7fcce99a1640 1 -- 172.21.15.40:0/135104967 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccdc0e15b0 msgr2=0x7fccdc0d9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7fcce99a1640 1 --2- 172.21.15.40:0/135104967 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccdc0e15b0 0x7fccdc0d9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7fcce99a1640 1 -- 172.21.15.40:0/135104967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccdc03a820 msgr2=0x7fccdc0db870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7fcce99a1640 1 --2- 172.21.15.40:0/135104967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccdc03a820 0x7fccdc0db870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7fcce99a1640 1 -- 172.21.15.40:0/135104967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcce00094d0 con 0x7fccdc0e11d0 2023-10-16T08:38:52.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7f1abd854640 1 Processor -- start 2023-10-16T08:38:52.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7fcce99a1640 1 --2- 172.21.15.40:0/135104967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc0e11d0 0x7fccdc036240 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcce00ac000 tx=0x7fcce001ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7fccdaffd640 1 -- 172.21.15.40:0/135104967 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce00ab600 con 0x7fccdc0e11d0 2023-10-16T08:38:52.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7fccdaffd640 1 -- 172.21.15.40:0/135104967 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcce00b2c40 con 0x7fccdc0e11d0 2023-10-16T08:38:52.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7fccdaffd640 1 -- 172.21.15.40:0/135104967 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcce00be3f0 con 0x7fccdc0e11d0 2023-10-16T08:38:52.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7fcceb1a4640 1 -- 172.21.15.40:0/135104967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccdc0dad90 con 0x7fccdc0e11d0 2023-10-16T08:38:52.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7f1abd854640 1 -- start start 2023-10-16T08:38:52.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7f1abd854640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1ab8156a20 0x7f1ab8159e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7f1abd854640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ab80adb80 0x7f1ab80cc530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7f1abd854640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ab80cb370 0x7f1ab80cb800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7f1abd854640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab80cb0a0 con 0x7f1ab80adb80 2023-10-16T08:38:52.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7f1abd854640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab80cbd40 con 0x7f1ab8156a20 2023-10-16T08:38:52.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7f1abd854640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab8156e00 con 0x7f1ab80cb370 2023-10-16T08:38:52.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7f1ab67fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ab80adb80 0x7f1ab80cc530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7f1ab67fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1ab80adb80 0x7f1ab80cc530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58424/0 (socket says 172.21.15.40:58424) 2023-10-16T08:38:52.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7f1ab67fc640 1 -- 172.21.15.40:0/120033168 learned_addr learned my addr 172.21.15.40:0/120033168 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:52.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7f1ab6ffd640 1 --2- 172.21.15.40:0/120033168 >> v2:172.21.15.86:3300/0 conn(0x7f1ab8156a20 0x7f1ab8159e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7f1ab77fe640 1 --2- 172.21.15.40:0/120033168 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ab80cb370 0x7f1ab80cb800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.544+0000 7fcceb1a4640 1 -- 172.21.15.40:0/135104967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccdc0dbdb0 con 0x7fccdc0e11d0 2023-10-16T08:38:52.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.544+0000 7fccdaffd640 1 -- 172.21.15.40:0/135104967 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcce00be550 con 0x7fccdc0e11d0 2023-10-16T08:38:52.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.544+0000 7fcceb1a4640 1 -- 172.21.15.40:0/135104967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccdc0db590 con 0x7fccdc0e11d0 2023-10-16T08:38:52.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.544+0000 7fccdaffd640 1 --2- 172.21.15.40:0/135104967 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fccc80664a0 0x7fccc8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.544+0000 7fccdaffd640 1 -- 172.21.15.40:0/135104967 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcce011e2b0 con 0x7fccdc0e11d0 2023-10-16T08:38:52.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7f1ab6ffd640 1 -- 172.21.15.40:0/120033168 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ab80cb370 msgr2=0x7f1ab80cb800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:52.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7f1ab6ffd640 1 --2- 172.21.15.40:0/120033168 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ab80cb370 0x7f1ab80cb800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7f1ab6ffd640 1 -- 172.21.15.40:0/120033168 >> v2:172.21.15.40:3300/0 conn(0x7f1ab80adb80 msgr2=0x7f1ab80cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:52.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7f1ab6ffd640 1 --2- 172.21.15.40:0/120033168 >> v2:172.21.15.40:3300/0 conn(0x7f1ab80adb80 0x7f1ab80cc530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.540+0000 7f1ab6ffd640 1 -- 172.21.15.40:0/120033168 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab80ae0d0 con 0x7f1ab8156a20 2023-10-16T08:38:52.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.544+0000 7fccea1a2640 1 --2- 172.21.15.40:0/135104967 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fccc80664a0 0x7fccc8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.544+0000 7f1ab6ffd640 1 --2- 172.21.15.40:0/120033168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ab8156a20 0x7f1ab8159e60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80cb340 tx=0x7f1ab001ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91c1b3e1c10d0662 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.544+0000 7f1ab5ffb640 1 -- 172.21.15.40:0/120033168 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab00a77f0 con 0x7f1ab8156a20 2023-10-16T08:38:52.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.544+0000 7f1ab5ffb640 1 -- 172.21.15.40:0/120033168 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ab00ab070 con 0x7f1ab8156a20 2023-10-16T08:38:52.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.544+0000 7f1ab5ffb640 1 -- 172.21.15.40:0/120033168 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ab00a7b70 con 0x7f1ab8156a20 2023-10-16T08:38:52.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.544+0000 7fccea1a2640 1 --2- 172.21.15.40:0/135104967 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fccc80664a0 0x7fccc8068960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009a50 tx=0x7fccd4009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.544+0000 7f1abd854640 1 -- 172.21.15.40:0/120033168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ab8156a20 msgr2=0x7f1ab8159e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.544+0000 7f1abd854640 1 --2- 172.21.15.40:0/120033168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ab8156a20 0x7f1ab8159e60 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1ab80cb340 tx=0x7f1ab001ed30 comp rx=0 tx=0).stop 2023-10-16T08:38:52.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.544+0000 7f1abd854640 1 -- 172.21.15.40:0/120033168 shutdown_connections 2023-10-16T08:38:52.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.544+0000 7f1abd854640 1 --2- 172.21.15.40:0/120033168 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ab80cb370 0x7f1ab80cb800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.544+0000 7f1abd854640 1 --2- 172.21.15.40:0/120033168 >> v2:172.21.15.40:3300/0 conn(0x7f1ab80adb80 0x7f1ab80cc530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.544+0000 7f1abd854640 1 --2- 172.21.15.40:0/120033168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ab8156a20 0x7f1ab8159e60 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.544+0000 7f1abd854640 1 -- 172.21.15.40:0/120033168 >> 172.21.15.40:0/120033168 conn(0x7f1ab80b0470 msgr2=0x7f1ab80b0880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.544+0000 7f1abd854640 1 -- 172.21.15.40:0/120033168 shutdown_connections 2023-10-16T08:38:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.544+0000 7f1abd854640 1 -- 172.21.15.40:0/120033168 wait complete. 2023-10-16T08:38:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.548+0000 7f1abd854640 1 Processor -- start 2023-10-16T08:38:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.548+0000 7f1abd854640 1 -- start start 2023-10-16T08:38:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.548+0000 7f1abd854640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ab80adb80 0x7f1ab8155a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.548+0000 7f1abd854640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ab80cb370 0x7f1ab80b2100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.548+0000 7f1abd854640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab8156a20 0x7f1ab81582a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.548+0000 7f1abd854640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab80b2640 con 0x7f1ab8156a20 2023-10-16T08:38:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.548+0000 7f1abd854640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab80c8160 con 0x7f1ab80cb370 2023-10-16T08:38:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.548+0000 7f1abd854640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab80c8440 con 0x7f1ab80adb80 2023-10-16T08:38:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.548+0000 7f1ab77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab8156a20 0x7f1ab81582a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.548+0000 7f1ab77fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab8156a20 0x7f1ab81582a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58434/0 (socket says 172.21.15.40:58434) 2023-10-16T08:38:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.548+0000 7f1ab77fe640 1 -- 172.21.15.40:0/1675946826 learned_addr learned my addr 172.21.15.40:0/1675946826 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.548+0000 7f1ab6ffd640 1 --2- 172.21.15.40:0/1675946826 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ab80adb80 0x7f1ab8155a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.548+0000 7f1ab67fc640 1 --2- 172.21.15.40:0/1675946826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ab80cb370 0x7f1ab80b2100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.548+0000 7f1ab77fe640 1 -- 172.21.15.40:0/1675946826 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ab80adb80 msgr2=0x7f1ab8155a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.548+0000 7f1ab77fe640 1 --2- 172.21.15.40:0/1675946826 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ab80adb80 0x7f1ab8155a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.548+0000 7f1ab77fe640 1 -- 172.21.15.40:0/1675946826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ab80cb370 msgr2=0x7f1ab80b2100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.548+0000 7f1ab77fe640 1 --2- 172.21.15.40:0/1675946826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ab80cb370 0x7f1ab80b2100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.548+0000 7f1ab77fe640 1 -- 172.21.15.40:0/1675946826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab0007910 con 0x7f1ab8156a20 2023-10-16T08:38:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.548+0000 7f1ab6ffd640 1 --2- 172.21.15.40:0/1675946826 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ab80adb80 0x7f1ab8155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:38:52.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.548+0000 7f1ab77fe640 1 --2- 172.21.15.40:0/1675946826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab8156a20 0x7f1ab81582a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00e7f0 tx=0x7f1aac00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.552+0000 7fccdaffd640 1 -- 172.21.15.40:0/135104967 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fccdc0db590 con 0x7fccdc0e11d0 2023-10-16T08:38:52.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.552+0000 7f92f75c0640 1 Processor -- start 2023-10-16T08:38:52.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.552+0000 7f92f75c0640 1 -- start start 2023-10-16T08:38:52.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.552+0000 7f1a9ffff640 1 -- 172.21.15.40:0/1675946826 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac009530 con 0x7f1ab8156a20 2023-10-16T08:38:52.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.552+0000 7f1abd854640 1 -- 172.21.15.40:0/1675946826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab80b1620 con 0x7f1ab8156a20 2023-10-16T08:38:52.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.552+0000 7f1abd854640 1 -- 172.21.15.40:0/1675946826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab80b1be0 con 0x7f1ab8156a20 2023-10-16T08:38:52.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.556+0000 7f1a9ffff640 1 -- 172.21.15.40:0/1675946826 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aac00eea0 con 0x7f1ab8156a20 2023-10-16T08:38:52.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.556+0000 7f1abd854640 1 -- 172.21.15.40:0/1675946826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab804e9c0 con 0x7f1ab8156a20 2023-10-16T08:38:52.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.556+0000 7f1a9ffff640 1 -- 172.21.15.40:0/1675946826 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aac00cb60 con 0x7f1ab8156a20 2023-10-16T08:38:52.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.556+0000 7f1a9ffff640 1 -- 172.21.15.40:0/1675946826 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1aac00cd20 con 0x7f1ab8156a20 2023-10-16T08:38:52.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.552+0000 7f92f75c0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f92f00d1320 0x7f92f00af890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.556+0000 7f92f75c0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f00af1e0 con 0x7f92f00cf390 2023-10-16T08:38:52.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.556+0000 7f92f75c0640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92f00af670 con 0x7f92f00b22d0 2023-10-16T08:38:52.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.556+0000 7f92f75c0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92f00cfad0 con 0x7f92f00d1320 2023-10-16T08:38:52.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.556+0000 7f92f535c640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f92f00b22d0 0x7f92f00cec80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.40:47248/0 (socket says 172.21.15.40:47248) 2023-10-16T08:38:52.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.556+0000 7f92f535c640 1 -- 172.21.15.40:0/69339979 learned_addr learned my addr 172.21.15.40:0/69339979 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:52.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.556+0000 7f92f5b5d640 1 --2- 172.21.15.40:0/69339979 >> v2:172.21.15.40:3300/0 conn(0x7f92f00d1320 0x7f92f00af890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.556+0000 7f92e7fff640 1 -- 172.21.15.40:0/69339979 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2652664436 0 0) 0x7f92f00af670 con 0x7f92f00b22d0 2023-10-16T08:38:52.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.556+0000 7f92e7fff640 1 -- 172.21.15.40:0/69339979 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92dc003530 con 0x7f92f00b22d0 2023-10-16T08:38:52.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.556+0000 7f92f5b5d640 1 -- 172.21.15.40:0/69339979 >> v1:172.21.15.86:6789/0 conn(0x7f92f00b22d0 legacy=0x7f92f00cec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.556+0000 7f92f5b5d640 1 -- 172.21.15.40:0/69339979 >> v1:172.21.15.40:6789/0 conn(0x7f92f00cf390 legacy=0x7f92f00b0570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:52.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.556+0000 7f92f5b5d640 1 -- 172.21.15.40:0/69339979 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f006beb0 con 0x7f92f00d1320 2023-10-16T08:38:52.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f1a9ffff640 1 --2- 172.21.15.40:0/1675946826 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1aa4066520 0x7f1aa40689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f1a9ffff640 1 -- 172.21.15.40:0/1675946826 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1aac084820 con 0x7f1ab8156a20 2023-10-16T08:38:52.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f70919ed640 1 Processor -- start 2023-10-16T08:38:52.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f348b999640 1 Processor -- start 2023-10-16T08:38:52.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f70919ed640 1 -- start start 2023-10-16T08:38:52.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f348b999640 1 -- start start 2023-10-16T08:38:52.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f1ab6ffd640 1 --2- 172.21.15.40:0/1675946826 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1aa4066520 0x7f1aa40689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f348b999640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3484160820 0x7f3484065a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f348b999640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3484160c80 0x7f3484065fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f348b999640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34840cad30 con 0x7f348415a890 2023-10-16T08:38:52.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f348b999640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f348415ac70 con 0x7f3484160c80 2023-10-16T08:38:52.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f348b999640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3484161200 con 0x7f3484160820 2023-10-16T08:38:52.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f3488f34640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3484160820 0x7f3484065a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f3488f34640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3484160820 0x7f3484065a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.40:54738/0 (socket says 172.21.15.40:54738) 2023-10-16T08:38:52.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f3489f36640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3484160c80 0x7f3484065fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f3488f34640 1 -- 172.21.15.40:0/4059417741 learned_addr learned my addr 172.21.15.40:0/4059417741 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:52.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f1ab6ffd640 1 --2- 172.21.15.40:0/1675946826 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1aa4066520 0x7f1aa40689e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00b2000 tx=0x7f1ab000a040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f70919ed640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f70840df270 0x7f7084053ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f92f5b5d640 1 --2- 172.21.15.40:0/69339979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00d1320 0x7f92f00af890 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009d90 tx=0x7f92e801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06c46c0e784ff8a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f708bfff640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f7084053070 0x7f7084007100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.40:47274/0 (socket says 172.21.15.40:47274) 2023-10-16T08:38:52.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f708bfff640 1 -- 172.21.15.40:0/1953982576 learned_addr learned my addr 172.21.15.40:0/1953982576 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:52.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f70919ed640 1 -- 172.21.15.40:0/1953982576 --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7084005dc0 con 0x7f708404fa60 2023-10-16T08:38:52.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f70919ed640 1 -- 172.21.15.40:0/1953982576 --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7084006060 con 0x7f7084053070 2023-10-16T08:38:52.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f70919ed640 1 -- 172.21.15.40:0/1953982576 --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7084006340 con 0x7f70840df270 2023-10-16T08:38:52.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f70911ec640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f708404fa60 0x7f7084037fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58408/0 (socket says 172.21.15.40:58408) 2023-10-16T08:38:52.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f70909eb640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f70840df270 0x7f7084053ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f3488f34640 1 -- 172.21.15.40:0/4059417741 >> v2:172.21.15.86:3300/0 conn(0x7f3484160c80 msgr2=0x7f3484065fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:52.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f3488f34640 1 --2- 172.21.15.40:0/4059417741 >> v2:172.21.15.86:3300/0 conn(0x7f3484160c80 0x7f3484065fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f3488f34640 1 -- 172.21.15.40:0/4059417741 >> v1:172.21.15.86:6789/0 conn(0x7f348415a890 legacy=0x7f348415dc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f3488f34640 1 -- 172.21.15.40:0/4059417741 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34841614e0 con 0x7f3484160820 2023-10-16T08:38:52.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f708b7fe640 1 -- 172.21.15.40:0/1953982576 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2577732284 0 0) 0x7f7084006060 con 0x7f7084053070 2023-10-16T08:38:52.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.560+0000 7f708b7fe640 1 -- 172.21.15.40:0/1953982576 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f707c003530 con 0x7f7084053070 2023-10-16T08:38:52.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f708b7fe640 1 -- 172.21.15.40:0/1953982576 <== mon.1 v1:172.21.15.86:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4182276537 0 0) 0x7f707c003530 con 0x7f7084053070 2023-10-16T08:38:52.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f708b7fe640 1 -- 172.21.15.40:0/1953982576 >> v2:172.21.15.86:3300/0 conn(0x7f70840df270 msgr2=0x7f7084053ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:52.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f708b7fe640 1 --2- 172.21.15.40:0/1953982576 >> v2:172.21.15.86:3300/0 conn(0x7f70840df270 0x7f7084053ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f708b7fe640 1 -- 172.21.15.40:0/1953982576 >> v1:172.21.15.40:6789/0 conn(0x7f708404fa60 legacy=0x7f7084037fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f708b7fe640 1 -- 172.21.15.40:0/1953982576 --> v1:172.21.15.86:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7084006620 con 0x7f7084053070 2023-10-16T08:38:52.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f708b7fe640 1 -- 172.21.15.40:0/1953982576 <== mon.1 v1:172.21.15.86:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f708c068670 con 0x7f7084053070 2023-10-16T08:38:52.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f708b7fe640 1 -- 172.21.15.40:0/1953982576 >> v1:172.21.15.86:6789/0 conn(0x7f7084053070 legacy=0x7f7084007100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f708b7fe640 1 --2- 172.21.15.40:0/1953982576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c003980 0x7f707c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f708b7fe640 1 --2- 172.21.15.40:0/1953982576 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f707c004320 0x7f707c0047f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f708b7fe640 1 --2- 172.21.15.40:0/1953982576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f707c004ff0 0x7f707c00b900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f708b7fe640 1 -- 172.21.15.40:0/1953982576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7084006340 con 0x7f707c004ff0 2023-10-16T08:38:52.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f708b7fe640 1 -- 172.21.15.40:0/1953982576 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f707c014700 con 0x7f707c003980 2023-10-16T08:38:52.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f708b7fe640 1 -- 172.21.15.40:0/1953982576 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f707c014890 con 0x7f707c004320 2023-10-16T08:38:52.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f70911ec640 1 --2- 172.21.15.40:0/1953982576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f707c004ff0 0x7f707c00b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f708bfff640 1 --2- 172.21.15.40:0/1953982576 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f707c004320 0x7f707c0047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f70909eb640 1 --2- 172.21.15.40:0/1953982576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c003980 0x7f707c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f3488f34640 1 --2- 172.21.15.40:0/4059417741 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3484160820 0x7f3484065a80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3480009870 tx=0x7f348001c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39ea9527c8235445 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f347bfff640 1 -- 172.21.15.40:0/4059417741 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34800ac600 con 0x7f3484160820 2023-10-16T08:38:52.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f92e7fff640 1 -- 172.21.15.40:0/69339979 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e80a59a0 con 0x7f92f00d1320 2023-10-16T08:38:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f1a9ffff640 1 -- 172.21.15.40:0/1675946826 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f1aac051c80 con 0x7f1ab8156a20 2023-10-16T08:38:52.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f70911ec640 1 -- 172.21.15.40:0/1953982576 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f707c004320 msgr2=0x7f707c0047f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f70911ec640 1 --2- 172.21.15.40:0/1953982576 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f707c004320 0x7f707c0047f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f70911ec640 1 -- 172.21.15.40:0/1953982576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c003980 msgr2=0x7f707c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:52.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f70911ec640 1 --2- 172.21.15.40:0/1953982576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f707c003980 0x7f707c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f70911ec640 1 -- 172.21.15.40:0/1953982576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f707c003530 con 0x7f707c004ff0 2023-10-16T08:38:52.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f70911ec640 1 -- 172.21.15.40:0/1953982576 reap_dead start 2023-10-16T08:38:52.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f70911ec640 1 --2- 172.21.15.40:0/1953982576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f707c004ff0 0x7f707c00b900 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f70800017c0 tx=0x7f708001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b24555d1030fa07 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f708b7fe640 1 -- 172.21.15.40:0/1953982576 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70800049f0 con 0x7f707c004ff0 2023-10-16T08:38:52.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f708b7fe640 1 -- 172.21.15.40:0/1953982576 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70800a7070 con 0x7f707c004ff0 2023-10-16T08:38:52.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f708b7fe640 1 -- 172.21.15.40:0/1953982576 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7080004d90 con 0x7f707c004ff0 2023-10-16T08:38:52.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f70919ed640 1 -- 172.21.15.40:0/1953982576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f707c004ff0 msgr2=0x7f707c00b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f70919ed640 1 --2- 172.21.15.40:0/1953982576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f707c004ff0 0x7f707c00b900 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f70800017c0 tx=0x7f708001c400 comp rx=0 tx=0).stop 2023-10-16T08:38:52.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f70919ed640 1 -- 172.21.15.40:0/1953982576 shutdown_connections 2023-10-16T08:38:52.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f70919ed640 1 --2- 172.21.15.40:0/1953982576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f707c004ff0 0x7f707c00b900 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f70919ed640 1 -- 172.21.15.40:0/1953982576 >> 172.21.15.40:0/1953982576 conn(0x7f7084009fa0 msgr2=0x7f708400a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:52.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f70919ed640 1 -- 172.21.15.40:0/1953982576 shutdown_connections 2023-10-16T08:38:52.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f70919ed640 1 -- 172.21.15.40:0/1953982576 wait complete. 2023-10-16T08:38:52.589 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f70919ed640 1 Processor -- start 2023-10-16T08:38:52.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f70919ed640 1 -- start start 2023-10-16T08:38:52.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70919ed640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f707c004ff0 0x7f70840d1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70919ed640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f708404b1d0 0x7f70840df650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70919ed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70840db9e0 0x7f70840d2690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70919ed640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70840db5c0 con 0x7f707c004ff0 2023-10-16T08:38:52.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70919ed640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70840da940 con 0x7f70840db9e0 2023-10-16T08:38:52.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70919ed640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70840dac20 con 0x7f708404b1d0 2023-10-16T08:38:52.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70909eb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f707c004ff0 0x7f70840d1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70909eb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f707c004ff0 0x7f70840d1ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58462/0 (socket says 172.21.15.40:58462) 2023-10-16T08:38:52.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70909eb640 1 -- 172.21.15.40:0/790980661 learned_addr learned my addr 172.21.15.40:0/790980661 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:52.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70911ec640 1 --2- 172.21.15.40:0/790980661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70840db9e0 0x7f70840d2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f708bfff640 1 --2- 172.21.15.40:0/790980661 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f708404b1d0 0x7f70840df650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70909eb640 1 -- 172.21.15.40:0/790980661 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f708404b1d0 msgr2=0x7f70840df650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70909eb640 1 --2- 172.21.15.40:0/790980661 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f708404b1d0 0x7f70840df650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70909eb640 1 -- 172.21.15.40:0/790980661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70840db9e0 msgr2=0x7f70840d2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70909eb640 1 --2- 172.21.15.40:0/790980661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70840db9e0 0x7f70840d2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70909eb640 1 -- 172.21.15.40:0/790980661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70840daf00 con 0x7f707c004ff0 2023-10-16T08:38:52.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70911ec640 1 --2- 172.21.15.40:0/790980661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70840db9e0 0x7f70840d2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:38:52.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f347bfff640 1 -- 172.21.15.40:0/4059417741 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34800a8050 con 0x7f3484160820 2023-10-16T08:38:52.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f347bfff640 1 -- 172.21.15.40:0/4059417741 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34800ac980 con 0x7f3484160820 2023-10-16T08:38:52.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f348b999640 1 -- 172.21.15.40:0/4059417741 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3484160820 msgr2=0x7f3484065a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f348b999640 1 --2- 172.21.15.40:0/4059417741 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3484160820 0x7f3484065a80 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3480009870 tx=0x7f348001c4f0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f348b999640 1 -- 172.21.15.40:0/4059417741 shutdown_connections 2023-10-16T08:38:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f348b999640 1 --2- 172.21.15.40:0/4059417741 >> v2:172.21.15.86:3300/0 conn(0x7f3484160c80 0x7f3484065fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f348b999640 1 --2- 172.21.15.40:0/4059417741 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3484160820 0x7f3484065a80 secure :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f3480009870 tx=0x7f348001c4f0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f348b999640 1 -- 172.21.15.40:0/4059417741 >> 172.21.15.40:0/4059417741 conn(0x7f3484073390 msgr2=0x7f3484069e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f348b999640 1 -- 172.21.15.40:0/4059417741 shutdown_connections 2023-10-16T08:38:52.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f348b999640 1 -- 172.21.15.40:0/4059417741 wait complete. 2023-10-16T08:38:52.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f348b999640 1 Processor -- start 2023-10-16T08:38:52.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f348b999640 1 -- start start 2023-10-16T08:38:52.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f348b999640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348415a890 0x7f3484204da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f348b999640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3484160c80 0x7f34840c1c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f348b999640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3484158b20 0x7f3484158fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f348b999640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34841594e0 con 0x7f348415a890 2023-10-16T08:38:52.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f348b999640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34840c5fc0 con 0x7f3484158b20 2023-10-16T08:38:52.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f348b999640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34840c50d0 con 0x7f3484160c80 2023-10-16T08:38:52.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f3489735640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348415a890 0x7f3484204da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f3489735640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348415a890 0x7f3484204da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58452/0 (socket says 172.21.15.40:58452) 2023-10-16T08:38:52.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f3489735640 1 -- 172.21.15.40:0/3814396859 learned_addr learned my addr 172.21.15.40:0/3814396859 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:52.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f3489f36640 1 --2- 172.21.15.40:0/3814396859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3484158b20 0x7f3484158fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f3489735640 1 -- 172.21.15.40:0/3814396859 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3484160c80 msgr2=0x7f34840c1c90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-16T08:38:52.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f3489735640 1 --2- 172.21.15.40:0/3814396859 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3484160c80 0x7f34840c1c90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f3489735640 1 -- 172.21.15.40:0/3814396859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3484158b20 msgr2=0x7f3484158fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f3489735640 1 --2- 172.21.15.40:0/3814396859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3484158b20 0x7f3484158fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f3489735640 1 -- 172.21.15.40:0/3814396859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34800094d0 con 0x7f348415a890 2023-10-16T08:38:52.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f3489735640 1 --2- 172.21.15.40:0/3814396859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348415a890 0x7f3484204da0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f347400c500 tx=0x7f347400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f92e7fff640 1 -- 172.21.15.40:0/69339979 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92e80a5b00 con 0x7f92f00d1320 2023-10-16T08:38:52.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f92e7fff640 1 -- 172.21.15.40:0/69339979 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e80a5e70 con 0x7f92f00d1320 2023-10-16T08:38:52.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f92f75c0640 1 -- 172.21.15.40:0/69339979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00d1320 msgr2=0x7f92f00af890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f92f75c0640 1 --2- 172.21.15.40:0/69339979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00d1320 0x7f92f00af890 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f92e8009d90 tx=0x7f92e801c8e0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f92f75c0640 1 -- 172.21.15.40:0/69339979 shutdown_connections 2023-10-16T08:38:52.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f92f75c0640 1 --2- 172.21.15.40:0/69339979 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00d1320 0x7f92f00af890 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f92f75c0640 1 -- 172.21.15.40:0/69339979 >> 172.21.15.40:0/69339979 conn(0x7f92f0099fa0 msgr2=0x7f92f00d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f92f75c0640 1 -- 172.21.15.40:0/69339979 shutdown_connections 2023-10-16T08:38:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.564+0000 7f92f75c0640 1 -- 172.21.15.40:0/69339979 wait complete. 2023-10-16T08:38:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f92f75c0640 1 Processor -- start 2023-10-16T08:38:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f92f75c0640 1 -- start start 2023-10-16T08:38:52.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f92f75c0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92f00b22d0 0x7f92f014f620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f92f75c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00cf390 0x7f92f0159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f92f75c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f00d1320 0x7f92f0152890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f92f75c0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f00c7ce0 con 0x7f92f00cf390 2023-10-16T08:38:52.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f92f75c0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f00c7ef0 con 0x7f92f00d1320 2023-10-16T08:38:52.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f92f75c0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f00c81d0 con 0x7f92f00b22d0 2023-10-16T08:38:52.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f92f4b5b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00cf390 0x7f92f0159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f92f4b5b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00cf390 0x7f92f0159150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58458/0 (socket says 172.21.15.40:58458) 2023-10-16T08:38:52.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f92f4b5b640 1 -- 172.21.15.40:0/3044458612 learned_addr learned my addr 172.21.15.40:0/3044458612 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:52.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f92f5b5d640 1 --2- 172.21.15.40:0/3044458612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f00d1320 0x7f92f0152890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f92f535c640 1 --2- 172.21.15.40:0/3044458612 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92f00b22d0 0x7f92f014f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f92f4b5b640 1 -- 172.21.15.40:0/3044458612 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92f00b22d0 msgr2=0x7f92f014f620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f92f4b5b640 1 --2- 172.21.15.40:0/3044458612 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92f00b22d0 0x7f92f014f620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f92f4b5b640 1 -- 172.21.15.40:0/3044458612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f00d1320 msgr2=0x7f92f0152890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f92f4b5b640 1 --2- 172.21.15.40:0/3044458612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f00d1320 0x7f92f0152890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f92f4b5b640 1 -- 172.21.15.40:0/3044458612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e8009a70 con 0x7f92f00cf390 2023-10-16T08:38:52.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.568+0000 7f92f4b5b640 1 --2- 172.21.15.40:0/3044458612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00cf390 0x7f92f0159150 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00c490 tx=0x7f92ec001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70909eb640 1 --2- 172.21.15.40:0/790980661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f707c004ff0 0x7f70840d1ea0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7078011960 tx=0x7f7078011d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70897fa640 1 -- 172.21.15.40:0/790980661 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f707800fa30 con 0x7f707c004ff0 2023-10-16T08:38:52.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70897fa640 1 -- 172.21.15.40:0/790980661 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f707800fb90 con 0x7f707c004ff0 2023-10-16T08:38:52.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70919ed640 1 -- 172.21.15.40:0/790980661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70840db1e0 con 0x7f707c004ff0 2023-10-16T08:38:52.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70897fa640 1 -- 172.21.15.40:0/790980661 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7078014440 con 0x7f707c004ff0 2023-10-16T08:38:52.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70919ed640 1 -- 172.21.15.40:0/790980661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70840dfb90 con 0x7f707c004ff0 2023-10-16T08:38:52.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70919ed640 1 -- 172.21.15.40:0/790980661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70840db370 con 0x7f707c004ff0 2023-10-16T08:38:52.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f7bf9454640 1 Processor -- start 2023-10-16T08:38:52.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f70897fa640 1 -- 172.21.15.40:0/790980661 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f70780145a0 con 0x7f707c004ff0 2023-10-16T08:38:52.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f92e5ffb640 1 -- 172.21.15.40:0/3044458612 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec003600 con 0x7f92f00cf390 2023-10-16T08:38:52.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f92e5ffb640 1 -- 172.21.15.40:0/3044458612 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ec003780 con 0x7f92f00cf390 2023-10-16T08:38:52.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f92f75c0640 1 -- 172.21.15.40:0/3044458612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92f00c84b0 con 0x7f92f00cf390 2023-10-16T08:38:52.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f92e5ffb640 1 -- 172.21.15.40:0/3044458612 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ec003b70 con 0x7f92f00cf390 2023-10-16T08:38:52.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f92f75c0640 1 -- 172.21.15.40:0/3044458612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f00c8a70 con 0x7f92f00cf390 2023-10-16T08:38:52.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.572+0000 7f7bf9454640 1 -- start start 2023-10-16T08:38:52.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f7bf9454640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bec0e77c0 0x7f7bec007280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f7bf9454640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7bec047e30 0x7f7bec037d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f7bf9454640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7bec006df0 con 0x7f7bec0382a0 2023-10-16T08:38:52.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f7bf9454640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7bec054a50 con 0x7f7bec047e30 2023-10-16T08:38:52.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f7bf9454640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec054490 con 0x7f7bec0e77c0 2023-10-16T08:38:52.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f70897fa640 1 --2- 172.21.15.40:0/790980661 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f70700664a0 0x7f7070068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f7bf37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7bec047e30 0x7f7bec037d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f7bf37fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7bec047e30 0x7f7bec037d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58472/0 (socket says 172.21.15.40:58472) 2023-10-16T08:38:52.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f7bf37fe640 1 -- 172.21.15.40:0/3991938139 learned_addr learned my addr 172.21.15.40:0/3991938139 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:52.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f70897fa640 1 -- 172.21.15.40:0/790980661 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7078084ab0 con 0x7f707c004ff0 2023-10-16T08:38:52.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f708bfff640 1 --2- 172.21.15.40:0/790980661 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f70700664a0 0x7f7070068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f708bfff640 1 --2- 172.21.15.40:0/790980661 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f70700664a0 0x7f7070068960 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f708c000b70 tx=0x7f708c06c040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f7bf37fe640 1 -- 172.21.15.40:0/3991938139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bec0e77c0 msgr2=0x7f7bec007280 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-16T08:38:52.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f7bf37fe640 1 --2- 172.21.15.40:0/3991938139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bec0e77c0 0x7f7bec007280 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f7bf37fe640 1 -- 172.21.15.40:0/3991938139 >> v1:172.21.15.40:6789/0 conn(0x7f7bec0382a0 legacy=0x7f7bec052790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.576+0000 7f7bf37fe640 1 -- 172.21.15.40:0/3991938139 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7bec054770 con 0x7f7bec047e30 2023-10-16T08:38:52.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7f3479ffb640 1 -- 172.21.15.40:0/3814396859 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474005220 con 0x7f348415a890 2023-10-16T08:38:52.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7f3479ffb640 1 -- 172.21.15.40:0/3814396859 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3474005380 con 0x7f348415a890 2023-10-16T08:38:52.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7f348b999640 1 -- 172.21.15.40:0/3814396859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34840c53b0 con 0x7f348415a890 2023-10-16T08:38:52.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7f7bf37fe640 1 --2- 172.21.15.40:0/3991938139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bec047e30 0x7f7bec037d60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009870 tx=0x7f7be801ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e9c3f64ad9c17c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7f3479ffb640 1 -- 172.21.15.40:0/3814396859 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3474007270 con 0x7f348415a890 2023-10-16T08:38:52.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7f348b999640 1 -- 172.21.15.40:0/3814396859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34840c5970 con 0x7f348415a890 2023-10-16T08:38:52.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7fe92702b640 1 Processor -- start 2023-10-16T08:38:52.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7fe92702b640 1 -- start start 2023-10-16T08:38:52.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7f92f75c0640 1 -- 172.21.15.40:0/3044458612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f004e9c0 con 0x7f92f00cf390 2023-10-16T08:38:52.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7fe92702b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe92006d2a0 0x7fe920158b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7fe92702b640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe9200adc40 0x7fe92006f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7fe92702b640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe92015d620 con 0x7fe9200ad7c0 2023-10-16T08:38:52.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7fe92702b640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe920159050 con 0x7fe92006d2a0 2023-10-16T08:38:52.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7fe92702b640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe92006af60 con 0x7fe9200adc40 2023-10-16T08:38:52.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7fe926029640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe92006d2a0 0x7fe920158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7fe926029640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe92006d2a0 0x7fe920158b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58480/0 (socket says 172.21.15.40:58480) 2023-10-16T08:38:52.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7fe926029640 1 -- 172.21.15.40:0/1293148066 learned_addr learned my addr 172.21.15.40:0/1293148066 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:52.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7fe92682a640 1 --2- 172.21.15.40:0/1293148066 >> v2:172.21.15.86:3300/0 conn(0x7fe9200adc40 0x7fe92006f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7fe926029640 1 -- 172.21.15.40:0/1293148066 >> v2:172.21.15.86:3300/0 conn(0x7fe9200adc40 msgr2=0x7fe92006f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:52.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7fe926029640 1 --2- 172.21.15.40:0/1293148066 >> v2:172.21.15.86:3300/0 conn(0x7fe9200adc40 0x7fe92006f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7fe926029640 1 -- 172.21.15.40:0/1293148066 >> v1:172.21.15.86:6789/0 conn(0x7fe9200ad7c0 legacy=0x7fe9200b0080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.580+0000 7fe926029640 1 -- 172.21.15.40:0/1293148066 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe92015c730 con 0x7fe92006d2a0 2023-10-16T08:38:52.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.584+0000 7f348b999640 1 -- 172.21.15.40:0/3814396859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f348406ae20 con 0x7f348415a890 2023-10-16T08:38:52.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.584+0000 7f70897fa640 1 -- 172.21.15.40:0/790980661 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f70780520a0 con 0x7f707c004ff0 2023-10-16T08:38:52.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.584+0000 7fe926029640 1 --2- 172.21.15.40:0/1293148066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe92006d2a0 0x7fe920158b10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe9200ce5d0 tx=0x7fe918027890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d64bffd8d70496 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.584+0000 7f92e5ffb640 1 -- 172.21.15.40:0/3044458612 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f92ec003cd0 con 0x7f92f00cf390 2023-10-16T08:38:52.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.584+0000 7f92e5ffb640 1 --2- 172.21.15.40:0/3044458612 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f92d4066520 0x7f92d40689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.584+0000 7f7bf2ffd640 1 -- 172.21.15.40:0/3991938139 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be80a8a50 con 0x7f7bec047e30 2023-10-16T08:38:52.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.584+0000 7fe925027640 1 -- 172.21.15.40:0/1293148066 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9180b4610 con 0x7fe92006d2a0 2023-10-16T08:38:52.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.584+0000 7f92f535c640 1 --2- 172.21.15.40:0/3044458612 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f92d4066520 0x7f92d40689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.588+0000 7f92f535c640 1 --2- 172.21.15.40:0/3044458612 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f92d4066520 0x7f92d40689e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f92e00011f0 tx=0x7f92e0007d40 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.588+0000 7f92e5ffb640 1 -- 172.21.15.40:0/3044458612 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f92ec054a80 con 0x7f92f00cf390 2023-10-16T08:38:52.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.588+0000 7f7bf2ffd640 1 -- 172.21.15.40:0/3991938139 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be80a8bb0 con 0x7f7bec047e30 2023-10-16T08:38:52.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.588+0000 7f7bf2ffd640 1 -- 172.21.15.40:0/3991938139 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be80b0440 con 0x7f7bec047e30 2023-10-16T08:38:52.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.588+0000 7f7bf9454640 1 -- 172.21.15.40:0/3991938139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bec047e30 msgr2=0x7f7bec037d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.588+0000 7f7bf9454640 1 --2- 172.21.15.40:0/3991938139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bec047e30 0x7f7bec037d60 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f7be8009870 tx=0x7f7be801ed50 comp rx=0 tx=0).stop 2023-10-16T08:38:52.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.588+0000 7f7bf9454640 1 -- 172.21.15.40:0/3991938139 shutdown_connections 2023-10-16T08:38:52.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.588+0000 7f7bf9454640 1 --2- 172.21.15.40:0/3991938139 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bec047e30 0x7f7bec037d60 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.588+0000 7f7bf9454640 1 --2- 172.21.15.40:0/3991938139 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bec0e77c0 0x7f7bec007280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.588+0000 7f7bf9454640 1 -- 172.21.15.40:0/3991938139 >> 172.21.15.40:0/3991938139 conn(0x7f7bec054bc0 msgr2=0x7f7bec054fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:52.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.588+0000 7f7bf9454640 1 -- 172.21.15.40:0/3991938139 shutdown_connections 2023-10-16T08:38:52.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.588+0000 7fe925027640 1 -- 172.21.15.40:0/1293148066 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9180b4770 con 0x7fe92006d2a0 2023-10-16T08:38:52.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.588+0000 7f3479ffb640 1 -- 172.21.15.40:0/3814396859 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f34740073d0 con 0x7f348415a890 2023-10-16T08:38:52.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f3479ffb640 1 --2- 172.21.15.40:0/3814396859 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3468066520 0x7f34680689e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.588+0000 7f7bf9454640 1 -- 172.21.15.40:0/3991938139 wait complete. 2023-10-16T08:38:52.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f7bf9454640 1 Processor -- start 2023-10-16T08:38:52.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f3479ffb640 1 -- 172.21.15.40:0/3814396859 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3474007680 con 0x7f348415a890 2023-10-16T08:38:52.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7fe92702b640 1 -- 172.21.15.40:0/1293148066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe92006d2a0 msgr2=0x7fe920158b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7fe92702b640 1 --2- 172.21.15.40:0/1293148066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe92006d2a0 0x7fe920158b10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe9200ce5d0 tx=0x7fe918027890 comp rx=0 tx=0).stop 2023-10-16T08:38:52.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7fe92702b640 1 -- 172.21.15.40:0/1293148066 shutdown_connections 2023-10-16T08:38:52.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7fe92702b640 1 --2- 172.21.15.40:0/1293148066 >> v2:172.21.15.86:3300/0 conn(0x7fe9200adc40 0x7fe92006f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7fe92702b640 1 --2- 172.21.15.40:0/1293148066 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe92006d2a0 0x7fe920158b10 secure :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fe9200ce5d0 tx=0x7fe918027890 comp rx=0 tx=0).stop 2023-10-16T08:38:52.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7fe92702b640 1 -- 172.21.15.40:0/1293148066 >> 172.21.15.40:0/1293148066 conn(0x7fe9200cb080 msgr2=0x7fe9200cb490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:52.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f7bf9454640 1 -- start start 2023-10-16T08:38:52.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7fe92702b640 1 -- 172.21.15.40:0/1293148066 shutdown_connections 2023-10-16T08:38:52.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f3488f34640 1 --2- 172.21.15.40:0/3814396859 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3468066520 0x7f34680689e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f7bf9454640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bec0382a0 0x7f7bec0e4250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f7bf9454640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bec047e30 0x7f7bec0e5310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f7bf9454640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bec0e77c0 0x7f7bec0e5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f7bf9454640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec044ae0 con 0x7f7bec047e30 2023-10-16T08:38:52.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f7bf3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bec0382a0 0x7f7bec0e4250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f7bf3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bec0382a0 0x7f7bec0e4250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.40:54076/0 (socket says 172.21.15.40:54076) 2023-10-16T08:38:52.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f7bf9454640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec044dc0 con 0x7f7bec0382a0 2023-10-16T08:38:52.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f7bf9454640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7bec0450a0 con 0x7f7bec0e77c0 2023-10-16T08:38:52.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f7bf3fff640 1 -- 172.21.15.40:0/293538638 learned_addr learned my addr 172.21.15.40:0/293538638 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:52.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f7bf8c53640 1 --2- 172.21.15.40:0/293538638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bec0e77c0 0x7f7bec0e5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f3488f34640 1 --2- 172.21.15.40:0/3814396859 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3468066520 0x7f34680689e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f34800ae000 tx=0x7f3480008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7fe92702b640 1 -- 172.21.15.40:0/1293148066 wait complete. 2023-10-16T08:38:52.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7fe92702b640 1 Processor -- start 2023-10-16T08:38:52.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f7bf3fff640 1 -- 172.21.15.40:0/293538638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bec0e77c0 msgr2=0x7f7bec0e5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f7bf3fff640 1 --2- 172.21.15.40:0/293538638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bec0e77c0 0x7f7bec0e5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f7bf3fff640 1 -- 172.21.15.40:0/293538638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bec047e30 msgr2=0x7f7bec0e5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f7bf3fff640 1 --2- 172.21.15.40:0/293538638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bec047e30 0x7f7bec0e5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f7bf3fff640 1 -- 172.21.15.40:0/293538638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7be80094d0 con 0x7f7bec0382a0 2023-10-16T08:38:52.639 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f7bf3fff640 1 --2- 172.21.15.40:0/293538638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bec0382a0 0x7f7bec0e4250 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7be000b3e0 tx=0x7f7be000b7e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7f7bf0ff9640 1 -- 172.21.15.40:0/293538638 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be0011830 con 0x7f7bec0382a0 2023-10-16T08:38:52.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7f7bf9454640 1 -- 172.21.15.40:0/293538638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7bec045380 con 0x7f7bec0382a0 2023-10-16T08:38:52.640 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7f7bf9454640 1 -- 172.21.15.40:0/293538638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7bec0447e0 con 0x7f7bec0382a0 2023-10-16T08:38:52.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.592+0000 7fe92702b640 1 -- start start 2023-10-16T08:38:52.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe92702b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9200ad7c0 0x7fe9200c3f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe92702b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9201ff4b0 0x7fe92015a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.641 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe926029640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9200ad7c0 0x7fe9200c3f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe926029640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9200ad7c0 0x7fe9200c3f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58494/0 (socket says 172.21.15.40:58494) 2023-10-16T08:38:52.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe92702b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9201f43f0 0x7fe9201f4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.642 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7f7bf0ff9640 1 -- 172.21.15.40:0/293538638 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7be00119b0 con 0x7f7bec0382a0 2023-10-16T08:38:52.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe92702b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9201f4db0 con 0x7fe9200ad7c0 2023-10-16T08:38:52.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe92702b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9202005d0 con 0x7fe9201ff4b0 2023-10-16T08:38:52.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe92702b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9201ff9c0 con 0x7fe9201f43f0 2023-10-16T08:38:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe925828640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9201ff4b0 0x7fe92015a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe925828640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9201ff4b0 0x7fe92015a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.40:54086/0 (socket says 172.21.15.40:54086) 2023-10-16T08:38:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe925828640 1 -- 172.21.15.40:0/1025078948 learned_addr learned my addr 172.21.15.40:0/1025078948 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:52.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe92682a640 1 --2- 172.21.15.40:0/1025078948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9201f43f0 0x7fe9201f4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe925828640 1 -- 172.21.15.40:0/1025078948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9201f43f0 msgr2=0x7fe9201f4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe925828640 1 --2- 172.21.15.40:0/1025078948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9201f43f0 0x7fe9201f4870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe925828640 1 -- 172.21.15.40:0/1025078948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9200ad7c0 msgr2=0x7fe9200c3f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe925828640 1 --2- 172.21.15.40:0/1025078948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9200ad7c0 0x7fe9200c3f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.645 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe925828640 1 -- 172.21.15.40:0/1025078948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9180140e0 con 0x7fe9201ff4b0 2023-10-16T08:38:52.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe925828640 1 --2- 172.21.15.40:0/1025078948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9201ff4b0 0x7fe92015a4e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe91000c500 tx=0x7fe91000c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7f7bf0ff9640 1 -- 172.21.15.40:0/293538638 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7be0011d60 con 0x7f7bec0382a0 2023-10-16T08:38:52.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7f7bf9454640 1 -- 172.21.15.40:0/293538638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7bec049c90 con 0x7f7bec0382a0 2023-10-16T08:38:52.646 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7f7bf0ff9640 1 -- 172.21.15.40:0/293538638 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7be001a400 con 0x7f7bec0382a0 2023-10-16T08:38:52.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe916ffd640 1 -- 172.21.15.40:0/1025078948 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe910005220 con 0x7fe9201ff4b0 2023-10-16T08:38:52.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7f7bf0ff9640 1 --2- 172.21.15.40:0/293538638 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7bd80664a0 0x7f7bd8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7f7bf0ff9640 1 -- 172.21.15.40:0/293538638 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7be0083eb0 con 0x7f7bec0382a0 2023-10-16T08:38:52.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7f7bf37fe640 1 --2- 172.21.15.40:0/293538638 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7bd80664a0 0x7f7bd8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe916ffd640 1 -- 172.21.15.40:0/1025078948 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe910005380 con 0x7fe9201ff4b0 2023-10-16T08:38:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe916ffd640 1 -- 172.21.15.40:0/1025078948 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe910007270 con 0x7fe9201ff4b0 2023-10-16T08:38:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe92702b640 1 -- 172.21.15.40:0/1025078948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9201ffca0 con 0x7fe9201ff4b0 2023-10-16T08:38:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.596+0000 7fe92702b640 1 -- 172.21.15.40:0/1025078948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe920200260 con 0x7fe9201ff4b0 2023-10-16T08:38:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.600+0000 7fe92702b640 1 -- 172.21.15.40:0/1025078948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe920200780 con 0x7fe9201ff4b0 2023-10-16T08:38:52.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.600+0000 7f7bf37fe640 1 --2- 172.21.15.40:0/293538638 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7bd80664a0 0x7f7bd8068960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7be80b2000 tx=0x7f7be8004740 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.600+0000 7f92e5ffb640 1 -- 172.21.15.40:0/3044458612 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f92ec050dc0 con 0x7f92f00cf390 2023-10-16T08:38:52.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.608+0000 7f3479ffb640 1 -- 172.21.15.40:0/3814396859 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f34740518c0 con 0x7f348415a890 2023-10-16T08:38:52.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.612+0000 7f7bf0ff9640 1 -- 172.21.15.40:0/293538638 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f7be00513e0 con 0x7f7bec0382a0 2023-10-16T08:38:52.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.612+0000 7fe916ffd640 1 -- 172.21.15.40:0/1025078948 <== mon.1 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe9100073d0 con 0x7fe9201ff4b0 2023-10-16T08:38:52.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.612+0000 7fe916ffd640 1 --2- 172.21.15.40:0/1025078948 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe9040664d0 0x7fe904068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:52.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.612+0000 7fe916ffd640 1 -- 172.21.15.40:0/1025078948 <== mon.1 v2:172.21.15.86:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe910084340 con 0x7fe9201ff4b0 2023-10-16T08:38:52.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.612+0000 7fe926029640 1 --2- 172.21.15.40:0/1025078948 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe9040664d0 0x7fe904068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:52.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.616+0000 7fe926029640 1 --2- 172.21.15.40:0/1025078948 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe9040664d0 0x7fe904068990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe9180bb000 tx=0x7fe918007040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:52.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.624+0000 7fe916ffd640 1 -- 172.21.15.40:0/1025078948 <== mon.1 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fe9100517e0 con 0x7fe9201ff4b0 2023-10-16T08:38:52.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.820+0000 7f7a8c7b7640 1 -- 172.21.15.40:0/2583366813 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f7a6c0057e0 con 0x7f7a84158cd0 2023-10-16T08:38:52.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.824+0000 7f7a7affd640 1 -- 172.21.15.40:0/2583366813 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7a74054520 con 0x7f7a84158cd0 2023-10-16T08:38:52.827 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2023-10-16T08:38:52.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.824+0000 7f7a8c7b7640 1 -- 172.21.15.40:0/2583366813 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7a680664f0 msgr2=0x7f7a680689b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.824+0000 7f7a8c7b7640 1 --2- 172.21.15.40:0/2583366813 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7a680664f0 0x7f7a680689b0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f7a7c0b2000 tx=0x7f7a7c00b040 comp rx=0 tx=0).stop 2023-10-16T08:38:52.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.824+0000 7f7a8c7b7640 1 -- 172.21.15.40:0/2583366813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a84158cd0 msgr2=0x7f7a840b2650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.824+0000 7f7a8c7b7640 1 --2- 172.21.15.40:0/2583366813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a84158cd0 0x7f7a840b2650 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f7a7400d800 tx=0x7f7a7400dcc0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.824+0000 7f7a8c7b7640 1 -- 172.21.15.40:0/2583366813 shutdown_connections 2023-10-16T08:38:52.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.824+0000 7f7a8c7b7640 1 --2- 172.21.15.40:0/2583366813 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7a680664f0 0x7f7a680689b0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.824+0000 7f7a8c7b7640 1 --2- 172.21.15.40:0/2583366813 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7a84158cd0 0x7f7a840b2650 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.824+0000 7f7a8c7b7640 1 --2- 172.21.15.40:0/2583366813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7a840b07a0 0x7f7a841f9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.824+0000 7f7a8c7b7640 1 --2- 172.21.15.40:0/2583366813 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7a8406c8a0 0x7f7a840b4030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.828+0000 7f7a8c7b7640 1 -- 172.21.15.40:0/2583366813 >> 172.21.15.40:0/2583366813 conn(0x7f7a840cb090 msgr2=0x7f7a840aed90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:52.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.828+0000 7f7a8c7b7640 1 -- 172.21.15.40:0/2583366813 shutdown_connections 2023-10-16T08:38:52.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.828+0000 7f7a8c7b7640 1 -- 172.21.15.40:0/2583366813 wait complete. 2023-10-16T08:38:52.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.844+0000 7fcceb1a4640 1 -- 172.21.15.40:0/135104967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7fccdc0db590 con 0x7fccdc0e11d0 2023-10-16T08:38:52.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.844+0000 7fccdaffd640 1 -- 172.21.15.40:0/135104967 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fccdc0db590 con 0x7fccdc0e11d0 2023-10-16T08:38:52.851 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-16T08:38:52.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.848+0000 7fcceb1a4640 1 -- 172.21.15.40:0/135104967 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fccc80664a0 msgr2=0x7fccc8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.848+0000 7fcceb1a4640 1 --2- 172.21.15.40:0/135104967 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fccc80664a0 0x7fccc8068960 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fccd4009a50 tx=0x7fccd4009190 comp rx=0 tx=0).stop 2023-10-16T08:38:52.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.848+0000 7fcceb1a4640 1 -- 172.21.15.40:0/135104967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc0e11d0 msgr2=0x7fccdc036240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.848+0000 7fcceb1a4640 1 --2- 172.21.15.40:0/135104967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc0e11d0 0x7fccdc036240 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fcce00ac000 tx=0x7fcce001ad50 comp rx=0 tx=0).stop 2023-10-16T08:38:52.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.848+0000 7fcceb1a4640 1 -- 172.21.15.40:0/135104967 shutdown_connections 2023-10-16T08:38:52.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.848+0000 7fcceb1a4640 1 --2- 172.21.15.40:0/135104967 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fccdc0e15b0 0x7fccdc0d9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.848+0000 7fcceb1a4640 1 --2- 172.21.15.40:0/135104967 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fccc80664a0 0x7fccc8068960 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.848+0000 7fcceb1a4640 1 --2- 172.21.15.40:0/135104967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fccdc0e11d0 0x7fccdc036240 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.848+0000 7fcceb1a4640 1 --2- 172.21.15.40:0/135104967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fccdc03a820 0x7fccdc0db870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.848+0000 7fcceb1a4640 1 -- 172.21.15.40:0/135104967 >> 172.21.15.40:0/135104967 conn(0x7fccdc056dd0 msgr2=0x7fccdc0530b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:52.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.848+0000 7fcceb1a4640 1 -- 172.21.15.40:0/135104967 shutdown_connections 2023-10-16T08:38:52.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.848+0000 7fcceb1a4640 1 -- 172.21.15.40:0/135104967 wait complete. 2023-10-16T08:38:52.856 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.3 2023-10-16T08:38:52.856 DEBUG:teuthology.parallel:result is None 2023-10-16T08:38:52.874 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.1 2023-10-16T08:38:52.874 DEBUG:teuthology.parallel:result is None 2023-10-16T08:38:52.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.884+0000 7f1abd854640 1 -- 172.21.15.40:0/1675946826 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f1ab806cda0 con 0x7f1ab8156a20 2023-10-16T08:38:52.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.884+0000 7f1a9ffff640 1 -- 172.21.15.40:0/1675946826 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1aac017020 con 0x7f1ab8156a20 2023-10-16T08:38:52.888 INFO:teuthology.orchestra.run.smithi040.stdout:55834574856 2023-10-16T08:38:52.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.888+0000 7f1abd854640 1 -- 172.21.15.40:0/1675946826 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1aa4066520 msgr2=0x7f1aa40689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.888+0000 7f1abd854640 1 --2- 172.21.15.40:0/1675946826 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1aa4066520 0x7f1aa40689e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1ab00b2000 tx=0x7f1ab000a040 comp rx=0 tx=0).stop 2023-10-16T08:38:52.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.888+0000 7f1abd854640 1 -- 172.21.15.40:0/1675946826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab8156a20 msgr2=0x7f1ab81582a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.888+0000 7f1abd854640 1 --2- 172.21.15.40:0/1675946826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab8156a20 0x7f1ab81582a0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f1aac00e7f0 tx=0x7f1aac00ecb0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.888+0000 7f1abd854640 1 -- 172.21.15.40:0/1675946826 shutdown_connections 2023-10-16T08:38:52.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.888+0000 7f1abd854640 1 --2- 172.21.15.40:0/1675946826 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1aa4066520 0x7f1aa40689e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.888+0000 7f1abd854640 1 --2- 172.21.15.40:0/1675946826 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ab8156a20 0x7f1ab81582a0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.888+0000 7f1abd854640 1 --2- 172.21.15.40:0/1675946826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ab80cb370 0x7f1ab80b2100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.888+0000 7f1abd854640 1 --2- 172.21.15.40:0/1675946826 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ab80adb80 0x7f1ab8155a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.888+0000 7f1abd854640 1 -- 172.21.15.40:0/1675946826 >> 172.21.15.40:0/1675946826 conn(0x7f1ab80b0470 msgr2=0x7f1ab80b0850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:52.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.888+0000 7f1abd854640 1 -- 172.21.15.40:0/1675946826 shutdown_connections 2023-10-16T08:38:52.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.888+0000 7f1abd854640 1 -- 172.21.15.40:0/1675946826 wait complete. 2023-10-16T08:38:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.900+0000 7fe92702b640 1 -- 172.21.15.40:0/1025078948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fe920067570 con 0x7fe9201ff4b0 2023-10-16T08:38:52.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.900+0000 7fe916ffd640 1 -- 172.21.15.40:0/1025078948 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fe910054e00 con 0x7fe9201ff4b0 2023-10-16T08:38:52.904 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-16T08:38:52.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.900+0000 7f70919ed640 1 -- 172.21.15.40:0/790980661 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f708400aa40 con 0x7f707c004ff0 2023-10-16T08:38:52.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.900+0000 7f70897fa640 1 -- 172.21.15.40:0/790980661 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f70780556c0 con 0x7f707c004ff0 2023-10-16T08:38:52.906 INFO:teuthology.orchestra.run.smithi040.stdout:55834574855 2023-10-16T08:38:52.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.900+0000 7fe92702b640 1 -- 172.21.15.40:0/1025078948 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe9040664d0 msgr2=0x7fe904068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.900+0000 7fe92702b640 1 --2- 172.21.15.40:0/1025078948 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe9040664d0 0x7fe904068990 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe9180bb000 tx=0x7fe918007040 comp rx=0 tx=0).stop 2023-10-16T08:38:52.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.900+0000 7fe92702b640 1 -- 172.21.15.40:0/1025078948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9201ff4b0 msgr2=0x7fe92015a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.900+0000 7fe92702b640 1 --2- 172.21.15.40:0/1025078948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9201ff4b0 0x7fe92015a4e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe91000c500 tx=0x7fe91000c9c0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.900+0000 7fe92702b640 1 -- 172.21.15.40:0/1025078948 shutdown_connections 2023-10-16T08:38:52.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.900+0000 7fe92702b640 1 --2- 172.21.15.40:0/1025078948 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe9201f43f0 0x7fe9201f4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.900+0000 7fe92702b640 1 --2- 172.21.15.40:0/1025078948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe9201ff4b0 0x7fe92015a4e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.900+0000 7fe92702b640 1 --2- 172.21.15.40:0/1025078948 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe9040664d0 0x7fe904068990 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.900+0000 7fe92702b640 1 --2- 172.21.15.40:0/1025078948 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe9200ad7c0 0x7fe9200c3f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.900+0000 7fe92702b640 1 -- 172.21.15.40:0/1025078948 >> 172.21.15.40:0/1025078948 conn(0x7fe9200cb080 msgr2=0x7fe9200cb460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:52.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.900+0000 7fe92702b640 1 -- 172.21.15.40:0/1025078948 shutdown_connections 2023-10-16T08:38:52.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.900+0000 7fe92702b640 1 -- 172.21.15.40:0/1025078948 wait complete. 2023-10-16T08:38:52.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.904+0000 7f7bf9454640 1 -- 172.21.15.40:0/293538638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f7bec005390 con 0x7f7bec0382a0 2023-10-16T08:38:52.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.904+0000 7f7bf0ff9640 1 -- 172.21.15.40:0/293538638 <== mon.1 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7be0015090 con 0x7f7bec0382a0 2023-10-16T08:38:52.910 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-16T08:38:52.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.904+0000 7f70919ed640 1 -- 172.21.15.40:0/790980661 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f70700664a0 msgr2=0x7f7070068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.904+0000 7f70919ed640 1 --2- 172.21.15.40:0/790980661 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f70700664a0 0x7f7070068960 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f708c000b70 tx=0x7f708c06c040 comp rx=0 tx=0).stop 2023-10-16T08:38:52.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.904+0000 7f70919ed640 1 -- 172.21.15.40:0/790980661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f707c004ff0 msgr2=0x7f70840d1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.904+0000 7f70919ed640 1 --2- 172.21.15.40:0/790980661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f707c004ff0 0x7f70840d1ea0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f7078011960 tx=0x7f7078011d60 comp rx=0 tx=0).stop 2023-10-16T08:38:52.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.904+0000 7f70919ed640 1 -- 172.21.15.40:0/790980661 shutdown_connections 2023-10-16T08:38:52.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.904+0000 7f70919ed640 1 --2- 172.21.15.40:0/790980661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70840db9e0 0x7f70840d2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.904+0000 7f70919ed640 1 --2- 172.21.15.40:0/790980661 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f708404b1d0 0x7f70840df650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.904+0000 7f70919ed640 1 --2- 172.21.15.40:0/790980661 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f70700664a0 0x7f7070068960 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.904+0000 7f70919ed640 1 --2- 172.21.15.40:0/790980661 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f707c004ff0 0x7f70840d1ea0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.904+0000 7f70919ed640 1 -- 172.21.15.40:0/790980661 >> 172.21.15.40:0/790980661 conn(0x7f7084009fa0 msgr2=0x7f70840de290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:52.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.904+0000 7f70919ed640 1 -- 172.21.15.40:0/790980661 shutdown_connections 2023-10-16T08:38:52.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.904+0000 7f70919ed640 1 -- 172.21.15.40:0/790980661 wait complete. 2023-10-16T08:38:52.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.908+0000 7f7bf9454640 1 -- 172.21.15.40:0/293538638 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7bd80664a0 msgr2=0x7f7bd8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.908+0000 7f7bf9454640 1 --2- 172.21.15.40:0/293538638 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7bd80664a0 0x7f7bd8068960 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f7be80b2000 tx=0x7f7be8004740 comp rx=0 tx=0).stop 2023-10-16T08:38:52.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.908+0000 7f7bf9454640 1 -- 172.21.15.40:0/293538638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bec0382a0 msgr2=0x7f7bec0e4250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.908+0000 7f7bf9454640 1 --2- 172.21.15.40:0/293538638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bec0382a0 0x7f7bec0e4250 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7be000b3e0 tx=0x7f7be000b7e0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.908+0000 7f7bf9454640 1 -- 172.21.15.40:0/293538638 shutdown_connections 2023-10-16T08:38:52.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.908+0000 7f7bf9454640 1 --2- 172.21.15.40:0/293538638 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7bec0e77c0 0x7f7bec0e5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.908+0000 7f7bf9454640 1 --2- 172.21.15.40:0/293538638 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7bd80664a0 0x7f7bd8068960 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.908+0000 7f7bf9454640 1 --2- 172.21.15.40:0/293538638 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7bec047e30 0x7f7bec0e5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.908+0000 7f7bf9454640 1 --2- 172.21.15.40:0/293538638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7bec0382a0 0x7f7bec0e4250 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.908+0000 7f7bf9454640 1 -- 172.21.15.40:0/293538638 >> 172.21.15.40:0/293538638 conn(0x7f7bec054bc0 msgr2=0x7f7bec04bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:52.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.908+0000 7f7bf9454640 1 -- 172.21.15.40:0/293538638 shutdown_connections 2023-10-16T08:38:52.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.908+0000 7f7bf9454640 1 -- 172.21.15.40:0/293538638 wait complete. 2023-10-16T08:38:52.917 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2023-10-16T08:38:52.917 DEBUG:teuthology.parallel:result is None 2023-10-16T08:38:52.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.916+0000 7f348b999640 1 -- 172.21.15.40:0/3814396859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f34840cad30 con 0x7f348415a890 2023-10-16T08:38:52.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.920+0000 7f3479ffb640 1 -- 172.21.15.40:0/3814396859 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f34840cad30 con 0x7f348415a890 2023-10-16T08:38:52.923 INFO:teuthology.orchestra.run.smithi040.stdout:55834574856 2023-10-16T08:38:52.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.920+0000 7f348b999640 1 -- 172.21.15.40:0/3814396859 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3468066520 msgr2=0x7f34680689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.920+0000 7f348b999640 1 --2- 172.21.15.40:0/3814396859 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3468066520 0x7f34680689e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f34800ae000 tx=0x7f3480008040 comp rx=0 tx=0).stop 2023-10-16T08:38:52.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.920+0000 7f348b999640 1 -- 172.21.15.40:0/3814396859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348415a890 msgr2=0x7f3484204da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.920+0000 7f348b999640 1 --2- 172.21.15.40:0/3814396859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348415a890 0x7f3484204da0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f347400c500 tx=0x7f347400c9c0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.920+0000 7f348b999640 1 -- 172.21.15.40:0/3814396859 shutdown_connections 2023-10-16T08:38:52.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.920+0000 7f348b999640 1 --2- 172.21.15.40:0/3814396859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3484158b20 0x7f3484158fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.920+0000 7f348b999640 1 --2- 172.21.15.40:0/3814396859 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3484160c80 0x7f34840c1c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.920+0000 7f348b999640 1 --2- 172.21.15.40:0/3814396859 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3468066520 0x7f34680689e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.920+0000 7f348b999640 1 --2- 172.21.15.40:0/3814396859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f348415a890 0x7f3484204da0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.920+0000 7f348b999640 1 -- 172.21.15.40:0/3814396859 >> 172.21.15.40:0/3814396859 conn(0x7f3484073390 msgr2=0x7f348415fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:52.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.920+0000 7f348b999640 1 -- 172.21.15.40:0/3814396859 shutdown_connections 2023-10-16T08:38:52.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.920+0000 7f348b999640 1 -- 172.21.15.40:0/3814396859 wait complete. 2023-10-16T08:38:52.928 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.0 2023-10-16T08:38:52.928 DEBUG:teuthology.parallel:result is None 2023-10-16T08:38:52.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.924+0000 7f92f75c0640 1 -- 172.21.15.40:0/3044458612 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f92f015f7f0 con 0x7f92f00cf390 2023-10-16T08:38:52.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.924+0000 7f92e5ffb640 1 -- 172.21.15.40:0/3044458612 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f92ec017030 con 0x7f92f00cf390 2023-10-16T08:38:52.929 INFO:teuthology.orchestra.run.smithi040.stdout:55834574854 2023-10-16T08:38:52.932 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.7 2023-10-16T08:38:52.932 DEBUG:teuthology.parallel:result is None 2023-10-16T08:38:52.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.928+0000 7f92f75c0640 1 -- 172.21.15.40:0/3044458612 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f92d4066520 msgr2=0x7f92d40689e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.928+0000 7f92f75c0640 1 --2- 172.21.15.40:0/3044458612 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f92d4066520 0x7f92d40689e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f92e00011f0 tx=0x7f92e0007d40 comp rx=0 tx=0).stop 2023-10-16T08:38:52.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.928+0000 7f92f75c0640 1 -- 172.21.15.40:0/3044458612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00cf390 msgr2=0x7f92f0159150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:52.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.928+0000 7f92f75c0640 1 --2- 172.21.15.40:0/3044458612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00cf390 0x7f92f0159150 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f92ec00c490 tx=0x7f92ec001e80 comp rx=0 tx=0).stop 2023-10-16T08:38:52.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.928+0000 7f92f75c0640 1 -- 172.21.15.40:0/3044458612 shutdown_connections 2023-10-16T08:38:52.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.928+0000 7f92f75c0640 1 --2- 172.21.15.40:0/3044458612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92f00d1320 0x7f92f0152890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.928+0000 7f92f75c0640 1 --2- 172.21.15.40:0/3044458612 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f92d4066520 0x7f92d40689e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.928+0000 7f92f75c0640 1 --2- 172.21.15.40:0/3044458612 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f92f00cf390 0x7f92f0159150 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.928+0000 7f92f75c0640 1 --2- 172.21.15.40:0/3044458612 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f92f00b22d0 0x7f92f014f620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:52.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.928+0000 7f92f75c0640 1 -- 172.21.15.40:0/3044458612 >> 172.21.15.40:0/3044458612 conn(0x7f92f0099fa0 msgr2=0x7f92f00b0570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:52.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.928+0000 7f92f75c0640 1 -- 172.21.15.40:0/3044458612 shutdown_connections 2023-10-16T08:38:52.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:52.928+0000 7f92f75c0640 1 -- 172.21.15.40:0/3044458612 wait complete. 2023-10-16T08:38:52.938 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.4 2023-10-16T08:38:52.938 DEBUG:teuthology.parallel:result is None 2023-10-16T08:38:52.944 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2023-10-16T08:38:52.945 DEBUG:teuthology.parallel:result is None 2023-10-16T08:38:52.950 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574854 for osd.6 2023-10-16T08:38:52.951 DEBUG:teuthology.parallel:result is None 2023-10-16T08:38:52.951 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-16T08:38:52.951 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:38:53.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.096+0000 7fbb1728e640 1 Processor -- start 2023-10-16T08:38:53.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.096+0000 7fbb1728e640 1 -- start start 2023-10-16T08:38:53.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1728e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb1015ca80 0x7fbb1006c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:53.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1728e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb100cc530 0x7fbb1006f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:53.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1728e640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb100cd640 con 0x7fbb1006fa20 2023-10-16T08:38:53.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1728e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb1006bc50 con 0x7fbb1015ca80 2023-10-16T08:38:53.104 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1728e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb100cca30 con 0x7fbb100cc530 2023-10-16T08:38:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1502a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb1015ca80 0x7fbb1006c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1582b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbb1006fa20 0x7fbb1006b540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58430/0 (socket says 172.21.15.40:58430) 2023-10-16T08:38:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1502a640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbb1015ca80 0x7fbb1006c7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58496/0 (socket says 172.21.15.40:58496) 2023-10-16T08:38:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1582b640 1 -- 172.21.15.40:0/3314988975 learned_addr learned my addr 172.21.15.40:0/3314988975 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:53.105 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb14829640 1 --2- 172.21.15.40:0/3314988975 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb100cc530 0x7fbb1006f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1502a640 1 -- 172.21.15.40:0/3314988975 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb100cc530 msgr2=0x7fbb1006f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1502a640 1 --2- 172.21.15.40:0/3314988975 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb100cc530 0x7fbb1006f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1502a640 1 -- 172.21.15.40:0/3314988975 >> v1:172.21.15.40:6789/0 conn(0x7fbb1006fa20 legacy=0x7fbb1006b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb03fff640 1 -- 172.21.15.40:0/3314988975 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2018702591 0 0) 0x7fbb100cd640 con 0x7fbb1006fa20 2023-10-16T08:38:53.106 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1502a640 1 -- 172.21.15.40:0/3314988975 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb100ccd10 con 0x7fbb1015ca80 2023-10-16T08:38:53.107 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb14829640 1 --2- 172.21.15.40:0/3314988975 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb100cc530 0x7fbb1006f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:53.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1502a640 1 --2- 172.21.15.40:0/3314988975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb1015ca80 0x7fbb1006c7e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00ba60 tx=0x7fbb0c01eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e18d1f946a92bc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:53.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb03fff640 1 -- 172.21.15.40:0/3314988975 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c0a7c10 con 0x7fbb1015ca80 2023-10-16T08:38:53.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb03fff640 1 -- 172.21.15.40:0/3314988975 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0c0a7d70 con 0x7fbb1015ca80 2023-10-16T08:38:53.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb03fff640 1 -- 172.21.15.40:0/3314988975 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c0af660 con 0x7fbb1015ca80 2023-10-16T08:38:53.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1728e640 1 -- 172.21.15.40:0/3314988975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb1015ca80 msgr2=0x7fbb1006c7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:53.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1728e640 1 --2- 172.21.15.40:0/3314988975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb1015ca80 0x7fbb1006c7e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c00ba60 tx=0x7fbb0c01eb40 comp rx=0 tx=0).stop 2023-10-16T08:38:53.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1728e640 1 -- 172.21.15.40:0/3314988975 shutdown_connections 2023-10-16T08:38:53.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1728e640 1 --2- 172.21.15.40:0/3314988975 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb100cc530 0x7fbb1006f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1728e640 1 --2- 172.21.15.40:0/3314988975 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb1015ca80 0x7fbb1006c7e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1728e640 1 -- 172.21.15.40:0/3314988975 >> 172.21.15.40:0/3314988975 conn(0x7fbb100ce960 msgr2=0x7fbb100ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:53.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1728e640 1 -- 172.21.15.40:0/3314988975 shutdown_connections 2023-10-16T08:38:53.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.100+0000 7fbb1728e640 1 -- 172.21.15.40:0/3314988975 wait complete. 2023-10-16T08:38:53.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb1728e640 1 Processor -- start 2023-10-16T08:38:53.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb1728e640 1 -- start start 2023-10-16T08:38:53.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb1728e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb100cc530 0x7fbb1014aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:53.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb1728e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb1015ca80 0x7fbb101f5850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:53.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb1728e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb100c5300 0x7fbb101f5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:53.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb1728e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb100c7ed0 con 0x7fbb100cc530 2023-10-16T08:38:53.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb1728e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb100c57a0 con 0x7fbb100c5300 2023-10-16T08:38:53.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb1728e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb100c7530 con 0x7fbb1015ca80 2023-10-16T08:38:53.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb1502a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb100cc530 0x7fbb1014aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:53.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb1502a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb100cc530 0x7fbb1014aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58512/0 (socket says 172.21.15.40:58512) 2023-10-16T08:38:53.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb1502a640 1 -- 172.21.15.40:0/4282567770 learned_addr learned my addr 172.21.15.40:0/4282567770 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:53.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb1502a640 1 -- 172.21.15.40:0/4282567770 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb1015ca80 msgr2=0x7fbb101f5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:53.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb1502a640 1 --2- 172.21.15.40:0/4282567770 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb1015ca80 0x7fbb101f5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb1502a640 1 -- 172.21.15.40:0/4282567770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb100c5300 msgr2=0x7fbb101f5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:53.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb1502a640 1 --2- 172.21.15.40:0/4282567770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb100c5300 0x7fbb101f5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb1502a640 1 -- 172.21.15.40:0/4282567770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0c00b6c0 con 0x7fbb100cc530 2023-10-16T08:38:53.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb1502a640 1 --2- 172.21.15.40:0/4282567770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb100cc530 0x7fbb1014aeb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c01c770 tx=0x7fbb0c01b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:53.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb01ffb640 1 -- 172.21.15.40:0/4282567770 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c0afe40 con 0x7fbb100cc530 2023-10-16T08:38:53.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb01ffb640 1 -- 172.21.15.40:0/4282567770 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0c0ba470 con 0x7fbb100cc530 2023-10-16T08:38:53.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb1728e640 1 -- 172.21.15.40:0/4282567770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb100c7810 con 0x7fbb100cc530 2023-10-16T08:38:53.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb01ffb640 1 -- 172.21.15.40:0/4282567770 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c0ba820 con 0x7fbb100cc530 2023-10-16T08:38:53.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb1728e640 1 -- 172.21.15.40:0/4282567770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb100c82f0 con 0x7fbb100cc530 2023-10-16T08:38:53.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb1728e640 1 -- 172.21.15.40:0/4282567770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1004e9c0 con 0x7fbb100cc530 2023-10-16T08:38:53.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.104+0000 7fbb01ffb640 1 -- 172.21.15.40:0/4282567770 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbb0c0b6070 con 0x7fbb100cc530 2023-10-16T08:38:53.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.108+0000 7fbb01ffb640 1 --2- 172.21.15.40:0/4282567770 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbaf40664a0 0x7fbaf4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:53.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.108+0000 7fbb01ffb640 1 -- 172.21.15.40:0/4282567770 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb0c11f190 con 0x7fbb100cc530 2023-10-16T08:38:53.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.108+0000 7fbb14829640 1 --2- 172.21.15.40:0/4282567770 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbaf40664a0 0x7fbaf4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:53.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.108+0000 7fbb14829640 1 --2- 172.21.15.40:0/4282567770 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbaf40664a0 0x7fbaf4068960 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbb10165060 tx=0x7fbb04008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:53.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.112+0000 7fbb01ffb640 1 -- 172.21.15.40:0/4282567770 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fbb0c0ec610 con 0x7fbb100cc530 2023-10-16T08:38:53.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.260+0000 7fbb1728e640 1 -- 172.21.15.40:0/4282567770 --> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbb100cdad0 con 0x7fbaf40664a0 2023-10-16T08:38:53.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.280+0000 7fbb01ffb640 1 -- 172.21.15.40:0/4282567770 <== mgr.4111 v2:172.21.15.86:6832/1188970899 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264499 (secure 0 0 0) 0x7fbb0400a4c0 con 0x7fbaf40664a0 2023-10-16T08:38:53.287 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:53.289 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-16T08:38:53.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.284+0000 7fbb1728e640 1 -- 172.21.15.40:0/4282567770 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbaf40664a0 msgr2=0x7fbaf4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:53.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.284+0000 7fbb1728e640 1 --2- 172.21.15.40:0/4282567770 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbaf40664a0 0x7fbaf4068960 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fbb10165060 tx=0x7fbb04008040 comp rx=0 tx=0).stop 2023-10-16T08:38:53.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.284+0000 7fbb1728e640 1 -- 172.21.15.40:0/4282567770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb100cc530 msgr2=0x7fbb1014aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:53.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.284+0000 7fbb1728e640 1 --2- 172.21.15.40:0/4282567770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb100cc530 0x7fbb1014aeb0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c01c770 tx=0x7fbb0c01b8e0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.284+0000 7fbb1728e640 1 -- 172.21.15.40:0/4282567770 shutdown_connections 2023-10-16T08:38:53.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.284+0000 7fbb1728e640 1 --2- 172.21.15.40:0/4282567770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb100c5300 0x7fbb101f5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.284+0000 7fbb1728e640 1 --2- 172.21.15.40:0/4282567770 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb1015ca80 0x7fbb101f5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.284+0000 7fbb1728e640 1 --2- 172.21.15.40:0/4282567770 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbaf40664a0 0x7fbaf4068960 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.284+0000 7fbb1728e640 1 --2- 172.21.15.40:0/4282567770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb100cc530 0x7fbb1014aeb0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.284+0000 7fbb1728e640 1 -- 172.21.15.40:0/4282567770 >> 172.21.15.40:0/4282567770 conn(0x7fbb100ce960 msgr2=0x7fbb100cb3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:53.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.284+0000 7fbb1728e640 1 -- 172.21.15.40:0/4282567770 shutdown_connections 2023-10-16T08:38:53.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.284+0000 7fbb1728e640 1 -- 172.21.15.40:0/4282567770 wait complete. 2023-10-16T08:38:53.304 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-16T08:38:52.940370+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219016,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755704,"statfs":{"total":773094113280,"available":772869840896,"internally_reserved":0,"allocated":2891776,"data_stored":1752740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003099"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931534+0000","last_change":"2023-10-16T08:38:45.650619+0000","last_active":"2023-10-16T08:38:46.931534+0000","last_peered":"2023-10-16T08:38:46.931534+0000","last_clean":"2023-10-16T08:38:46.931534+0000","last_became_active":"2023-10-16T08:38:45.649704+0000","last_became_peered":"2023-10-16T08:38:45.649704+0000","last_unstale":"2023-10-16T08:38:46.931534+0000","last_undegraded":"2023-10-16T08:38:46.931534+0000","last_fullsized":"2023-10-16T08:38:46.931534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:58:51.599709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626698+0000","last_change":"2023-10-16T08:38:45.657659+0000","last_active":"2023-10-16T08:38:46.626698+0000","last_peered":"2023-10-16T08:38:46.626698+0000","last_clean":"2023-10-16T08:38:46.626698+0000","last_became_active":"2023-10-16T08:38:45.656884+0000","last_became_peered":"2023-10-16T08:38:45.656884+0000","last_unstale":"2023-10-16T08:38:46.626698+0000","last_undegraded":"2023-10-16T08:38:46.626698+0000","last_fullsized":"2023-10-16T08:38:46.626698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926261+0000","last_change":"2023-10-16T08:38:45.630125+0000","last_active":"2023-10-16T08:38:46.926261+0000","last_peered":"2023-10-16T08:38:46.926261+0000","last_clean":"2023-10-16T08:38:46.926261+0000","last_became_active":"2023-10-16T08:38:45.629297+0000","last_became_peered":"2023-10-16T08:38:45.629297+0000","last_unstale":"2023-10-16T08:38:46.926261+0000","last_undegraded":"2023-10-16T08:38:46.926261+0000","last_fullsized":"2023-10-16T08:38:46.926261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931028+0000","last_change":"2023-10-16T08:38:45.675996+0000","last_active":"2023-10-16T08:38:46.931028+0000","last_peered":"2023-10-16T08:38:46.931028+0000","last_clean":"2023-10-16T08:38:46.931028+0000","last_became_active":"2023-10-16T08:38:45.675327+0000","last_became_peered":"2023-10-16T08:38:45.675327+0000","last_unstale":"2023-10-16T08:38:46.931028+0000","last_undegraded":"2023-10-16T08:38:46.931028+0000","last_fullsized":"2023-10-16T08:38:46.931028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:33.190218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930419+0000","last_change":"2023-10-16T08:38:45.653550+0000","last_active":"2023-10-16T08:38:46.930419+0000","last_peered":"2023-10-16T08:38:46.930419+0000","last_clean":"2023-10-16T08:38:46.930419+0000","last_became_active":"2023-10-16T08:38:45.652808+0000","last_became_peered":"2023-10-16T08:38:45.652808+0000","last_unstale":"2023-10-16T08:38:46.930419+0000","last_undegraded":"2023-10-16T08:38:46.930419+0000","last_fullsized":"2023-10-16T08:38:46.930419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930519+0000","last_change":"2023-10-16T08:38:45.675086+0000","last_active":"2023-10-16T08:38:46.930519+0000","last_peered":"2023-10-16T08:38:46.930519+0000","last_clean":"2023-10-16T08:38:46.930519+0000","last_became_active":"2023-10-16T08:38:45.671146+0000","last_became_peered":"2023-10-16T08:38:45.671146+0000","last_unstale":"2023-10-16T08:38:46.930519+0000","last_undegraded":"2023-10-16T08:38:46.930519+0000","last_fullsized":"2023-10-16T08:38:46.930519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931145+0000","last_change":"2023-10-16T08:38:45.652595+0000","last_active":"2023-10-16T08:38:46.931145+0000","last_peered":"2023-10-16T08:38:46.931145+0000","last_clean":"2023-10-16T08:38:46.931145+0000","last_became_active":"2023-10-16T08:38:45.651848+0000","last_became_peered":"2023-10-16T08:38:45.651848+0000","last_unstale":"2023-10-16T08:38:46.931145+0000","last_undegraded":"2023-10-16T08:38:46.931145+0000","last_fullsized":"2023-10-16T08:38:46.931145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:13.453321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628116+0000","last_change":"2023-10-16T08:38:45.668858+0000","last_active":"2023-10-16T08:38:46.628116+0000","last_peered":"2023-10-16T08:38:46.628116+0000","last_clean":"2023-10-16T08:38:46.628116+0000","last_became_active":"2023-10-16T08:38:45.664702+0000","last_became_peered":"2023-10-16T08:38:45.664702+0000","last_unstale":"2023-10-16T08:38:46.628116+0000","last_undegraded":"2023-10-16T08:38:46.628116+0000","last_fullsized":"2023-10-16T08:38:46.628116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931734+0000","last_change":"2023-10-16T08:38:45.667037+0000","last_active":"2023-10-16T08:38:46.931734+0000","last_peered":"2023-10-16T08:38:46.931734+0000","last_clean":"2023-10-16T08:38:46.931734+0000","last_became_active":"2023-10-16T08:38:45.666474+0000","last_became_peered":"2023-10-16T08:38:45.666474+0000","last_unstale":"2023-10-16T08:38:46.931734+0000","last_undegraded":"2023-10-16T08:38:46.931734+0000","last_fullsized":"2023-10-16T08:38:46.931734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:11:02.634006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633786+0000","last_change":"2023-10-16T08:38:45.669859+0000","last_active":"2023-10-16T08:38:46.633786+0000","last_peered":"2023-10-16T08:38:46.633786+0000","last_clean":"2023-10-16T08:38:46.633786+0000","last_became_active":"2023-10-16T08:38:45.668278+0000","last_became_peered":"2023-10-16T08:38:45.668278+0000","last_unstale":"2023-10-16T08:38:46.633786+0000","last_undegraded":"2023-10-16T08:38:46.633786+0000","last_fullsized":"2023-10-16T08:38:46.633786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633464+0000","last_change":"2023-10-16T08:38:45.664567+0000","last_active":"2023-10-16T08:38:46.633464+0000","last_peered":"2023-10-16T08:38:46.633464+0000","last_clean":"2023-10-16T08:38:46.633464+0000","last_became_active":"2023-10-16T08:38:45.663813+0000","last_became_peered":"2023-10-16T08:38:45.663813+0000","last_unstale":"2023-10-16T08:38:46.633464+0000","last_undegraded":"2023-10-16T08:38:46.633464+0000","last_fullsized":"2023-10-16T08:38:46.633464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.932310+0000","last_change":"2023-10-16T08:38:45.675925+0000","last_active":"2023-10-16T08:38:46.932310+0000","last_peered":"2023-10-16T08:38:46.932310+0000","last_clean":"2023-10-16T08:38:46.932310+0000","last_became_active":"2023-10-16T08:38:45.675320+0000","last_became_peered":"2023-10-16T08:38:45.675320+0000","last_unstale":"2023-10-16T08:38:46.932310+0000","last_undegraded":"2023-10-16T08:38:46.932310+0000","last_fullsized":"2023-10-16T08:38:46.932310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:11.991837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628294+0000","last_change":"2023-10-16T08:38:45.668056+0000","last_active":"2023-10-16T08:38:46.628294+0000","last_peered":"2023-10-16T08:38:46.628294+0000","last_clean":"2023-10-16T08:38:46.628294+0000","last_became_active":"2023-10-16T08:38:45.667428+0000","last_became_peered":"2023-10-16T08:38:45.667428+0000","last_unstale":"2023-10-16T08:38:46.628294+0000","last_undegraded":"2023-10-16T08:38:46.628294+0000","last_fullsized":"2023-10-16T08:38:46.628294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.925984+0000","last_change":"2023-10-16T08:38:45.637518+0000","last_active":"2023-10-16T08:38:46.925984+0000","last_peered":"2023-10-16T08:38:46.925984+0000","last_clean":"2023-10-16T08:38:46.925984+0000","last_became_active":"2023-10-16T08:38:45.636785+0000","last_became_peered":"2023-10-16T08:38:45.636785+0000","last_unstale":"2023-10-16T08:38:46.925984+0000","last_undegraded":"2023-10-16T08:38:46.925984+0000","last_fullsized":"2023-10-16T08:38:46.925984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:11.870171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626028+0000","last_change":"2023-10-16T08:38:45.656433+0000","last_active":"2023-10-16T08:38:46.626028+0000","last_peered":"2023-10-16T08:38:46.626028+0000","last_clean":"2023-10-16T08:38:46.626028+0000","last_became_active":"2023-10-16T08:38:45.655395+0000","last_became_peered":"2023-10-16T08:38:45.655395+0000","last_unstale":"2023-10-16T08:38:46.626028+0000","last_undegraded":"2023-10-16T08:38:46.626028+0000","last_fullsized":"2023-10-16T08:38:46.626028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.929381+0000","last_change":"2023-10-16T08:38:45.664192+0000","last_active":"2023-10-16T08:38:46.929381+0000","last_peered":"2023-10-16T08:38:46.929381+0000","last_clean":"2023-10-16T08:38:46.929381+0000","last_became_active":"2023-10-16T08:38:45.663312+0000","last_became_peered":"2023-10-16T08:38:45.663312+0000","last_unstale":"2023-10-16T08:38:46.929381+0000","last_undegraded":"2023-10-16T08:38:46.929381+0000","last_fullsized":"2023-10-16T08:38:46.929381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930671+0000","last_change":"2023-10-16T08:38:45.679894+0000","last_active":"2023-10-16T08:38:46.930671+0000","last_peered":"2023-10-16T08:38:46.930671+0000","last_clean":"2023-10-16T08:38:46.930671+0000","last_became_active":"2023-10-16T08:38:45.676912+0000","last_became_peered":"2023-10-16T08:38:45.676912+0000","last_unstale":"2023-10-16T08:38:46.930671+0000","last_undegraded":"2023-10-16T08:38:46.930671+0000","last_fullsized":"2023-10-16T08:38:46.930671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927699+0000","last_change":"2023-10-16T08:38:45.631488+0000","last_active":"2023-10-16T08:38:46.927699+0000","last_peered":"2023-10-16T08:38:46.927699+0000","last_clean":"2023-10-16T08:38:46.927699+0000","last_became_active":"2023-10-16T08:38:45.630641+0000","last_became_peered":"2023-10-16T08:38:45.630641+0000","last_unstale":"2023-10-16T08:38:46.927699+0000","last_undegraded":"2023-10-16T08:38:46.927699+0000","last_fullsized":"2023-10-16T08:38:46.927699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:11.991837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931188+0000","last_change":"2023-10-16T08:38:45.650087+0000","last_active":"2023-10-16T08:38:46.931188+0000","last_peered":"2023-10-16T08:38:46.931188+0000","last_clean":"2023-10-16T08:38:46.931188+0000","last_became_active":"2023-10-16T08:38:45.649475+0000","last_became_peered":"2023-10-16T08:38:45.649475+0000","last_unstale":"2023-10-16T08:38:46.931188+0000","last_undegraded":"2023-10-16T08:38:46.931188+0000","last_fullsized":"2023-10-16T08:38:46.931188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:26.292137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633283+0000","last_change":"2023-10-16T08:38:45.667336+0000","last_active":"2023-10-16T08:38:46.633283+0000","last_peered":"2023-10-16T08:38:46.633283+0000","last_clean":"2023-10-16T08:38:46.633283+0000","last_became_active":"2023-10-16T08:38:45.665291+0000","last_became_peered":"2023-10-16T08:38:45.665291+0000","last_unstale":"2023-10-16T08:38:46.633283+0000","last_undegraded":"2023-10-16T08:38:46.633283+0000","last_fullsized":"2023-10-16T08:38:46.633283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926490+0000","last_change":"2023-10-16T08:38:45.654036+0000","last_active":"2023-10-16T08:38:46.926490+0000","last_peered":"2023-10-16T08:38:46.926490+0000","last_clean":"2023-10-16T08:38:46.926490+0000","last_became_active":"2023-10-16T08:38:45.652412+0000","last_became_peered":"2023-10-16T08:38:45.652412+0000","last_unstale":"2023-10-16T08:38:46.926490+0000","last_undegraded":"2023-10-16T08:38:46.926490+0000","last_fullsized":"2023-10-16T08:38:46.926490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930858+0000","last_change":"2023-10-16T08:38:45.677644+0000","last_active":"2023-10-16T08:38:46.930858+0000","last_peered":"2023-10-16T08:38:46.930858+0000","last_clean":"2023-10-16T08:38:46.930858+0000","last_became_active":"2023-10-16T08:38:45.676844+0000","last_became_peered":"2023-10-16T08:38:45.676844+0000","last_unstale":"2023-10-16T08:38:46.930858+0000","last_undegraded":"2023-10-16T08:38:46.930858+0000","last_fullsized":"2023-10-16T08:38:46.930858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628178+0000","last_change":"2023-10-16T08:38:45.663771+0000","last_active":"2023-10-16T08:38:46.628178+0000","last_peered":"2023-10-16T08:38:46.628178+0000","last_clean":"2023-10-16T08:38:46.628178+0000","last_became_active":"2023-10-16T08:38:45.662944+0000","last_became_peered":"2023-10-16T08:38:45.662944+0000","last_unstale":"2023-10-16T08:38:46.628178+0000","last_undegraded":"2023-10-16T08:38:46.628178+0000","last_fullsized":"2023-10-16T08:38:46.628178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.929737+0000","last_change":"2023-10-16T08:38:45.667398+0000","last_active":"2023-10-16T08:38:46.929737+0000","last_peered":"2023-10-16T08:38:46.929737+0000","last_clean":"2023-10-16T08:38:46.929737+0000","last_became_active":"2023-10-16T08:38:45.666641+0000","last_became_peered":"2023-10-16T08:38:45.666641+0000","last_unstale":"2023-10-16T08:38:46.929737+0000","last_undegraded":"2023-10-16T08:38:46.929737+0000","last_fullsized":"2023-10-16T08:38:46.929737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628798+0000","last_change":"2023-10-16T08:38:45.670526+0000","last_active":"2023-10-16T08:38:46.628798+0000","last_peered":"2023-10-16T08:38:46.628798+0000","last_clean":"2023-10-16T08:38:46.628798+0000","last_became_active":"2023-10-16T08:38:45.669734+0000","last_became_peered":"2023-10-16T08:38:45.669734+0000","last_unstale":"2023-10-16T08:38:46.628798+0000","last_undegraded":"2023-10-16T08:38:46.628798+0000","last_fullsized":"2023-10-16T08:38:46.628798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633809+0000","last_change":"2023-10-16T08:38:45.670429+0000","last_active":"2023-10-16T08:38:46.633809+0000","last_peered":"2023-10-16T08:38:46.633809+0000","last_clean":"2023-10-16T08:38:46.633809+0000","last_became_active":"2023-10-16T08:38:45.665972+0000","last_became_peered":"2023-10-16T08:38:45.665972+0000","last_unstale":"2023-10-16T08:38:46.633809+0000","last_undegraded":"2023-10-16T08:38:46.633809+0000","last_fullsized":"2023-10-16T08:38:46.633809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626436+0000","last_change":"2023-10-16T08:38:45.667047+0000","last_active":"2023-10-16T08:38:46.626436+0000","last_peered":"2023-10-16T08:38:46.626436+0000","last_clean":"2023-10-16T08:38:46.626436+0000","last_became_active":"2023-10-16T08:38:45.666403+0000","last_became_peered":"2023-10-16T08:38:45.666403+0000","last_unstale":"2023-10-16T08:38:46.626436+0000","last_undegraded":"2023-10-16T08:38:46.626436+0000","last_fullsized":"2023-10-16T08:38:46.626436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628642+0000","last_change":"2023-10-16T08:38:45.668007+0000","last_active":"2023-10-16T08:38:46.628642+0000","last_peered":"2023-10-16T08:38:46.628642+0000","last_clean":"2023-10-16T08:38:46.628642+0000","last_became_active":"2023-10-16T08:38:45.664572+0000","last_became_peered":"2023-10-16T08:38:45.664572+0000","last_unstale":"2023-10-16T08:38:46.628642+0000","last_undegraded":"2023-10-16T08:38:46.628642+0000","last_fullsized":"2023-10-16T08:38:46.628642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926222+0000","last_change":"2023-10-16T08:38:45.667378+0000","last_active":"2023-10-16T08:38:46.926222+0000","last_peered":"2023-10-16T08:38:46.926222+0000","last_clean":"2023-10-16T08:38:46.926222+0000","last_became_active":"2023-10-16T08:38:45.666547+0000","last_became_peered":"2023-10-16T08:38:45.666547+0000","last_unstale":"2023-10-16T08:38:46.926222+0000","last_undegraded":"2023-10-16T08:38:46.926222+0000","last_fullsized":"2023-10-16T08:38:46.926222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931201+0000","last_change":"2023-10-16T08:38:45.669071+0000","last_active":"2023-10-16T08:38:46.931201+0000","last_peered":"2023-10-16T08:38:46.931201+0000","last_clean":"2023-10-16T08:38:46.931201+0000","last_became_active":"2023-10-16T08:38:45.666591+0000","last_became_peered":"2023-10-16T08:38:45.666591+0000","last_unstale":"2023-10-16T08:38:46.931201+0000","last_undegraded":"2023-10-16T08:38:46.931201+0000","last_fullsized":"2023-10-16T08:38:46.931201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:02:39.840175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927293+0000","last_change":"2023-10-16T08:38:45.648288+0000","last_active":"2023-10-16T08:38:46.927293+0000","last_peered":"2023-10-16T08:38:46.927293+0000","last_clean":"2023-10-16T08:38:46.927293+0000","last_became_active":"2023-10-16T08:38:45.647484+0000","last_became_peered":"2023-10-16T08:38:45.647484+0000","last_unstale":"2023-10-16T08:38:46.927293+0000","last_undegraded":"2023-10-16T08:38:46.927293+0000","last_fullsized":"2023-10-16T08:38:46.927293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:20.779617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.632427+0000","last_change":"2023-10-16T08:38:45.638183+0000","last_active":"2023-10-16T08:38:46.632427+0000","last_peered":"2023-10-16T08:38:46.632427+0000","last_clean":"2023-10-16T08:38:46.632427+0000","last_became_active":"2023-10-16T08:38:45.636762+0000","last_became_peered":"2023-10-16T08:38:45.636762+0000","last_unstale":"2023-10-16T08:38:46.632427+0000","last_undegraded":"2023-10-16T08:38:46.632427+0000","last_fullsized":"2023-10-16T08:38:46.632427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.929995+0000","last_change":"2023-10-16T08:38:45.659464+0000","last_active":"2023-10-16T08:38:46.929995+0000","last_peered":"2023-10-16T08:38:46.929995+0000","last_clean":"2023-10-16T08:38:46.929995+0000","last_became_active":"2023-10-16T08:38:45.658808+0000","last_became_peered":"2023-10-16T08:38:45.658808+0000","last_unstale":"2023-10-16T08:38:46.929995+0000","last_undegraded":"2023-10-16T08:38:46.929995+0000","last_fullsized":"2023-10-16T08:38:46.929995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:25.386221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628419+0000","last_change":"2023-10-16T08:38:44.644122+0000","last_active":"2023-10-16T08:38:46.628419+0000","last_peered":"2023-10-16T08:38:46.628419+0000","last_clean":"2023-10-16T08:38:46.628419+0000","last_became_active":"2023-10-16T08:38:44.643614+0000","last_became_peered":"2023-10-16T08:38:44.643614+0000","last_unstale":"2023-10-16T08:38:46.628419+0000","last_undegraded":"2023-10-16T08:38:46.628419+0000","last_fullsized":"2023-10-16T08:38:46.628419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926570+0000","last_change":"2023-10-16T08:38:45.659854+0000","last_active":"2023-10-16T08:38:46.926570+0000","last_peered":"2023-10-16T08:38:46.926570+0000","last_clean":"2023-10-16T08:38:46.926570+0000","last_became_active":"2023-10-16T08:38:45.658409+0000","last_became_peered":"2023-10-16T08:38:45.658409+0000","last_unstale":"2023-10-16T08:38:46.926570+0000","last_undegraded":"2023-10-16T08:38:46.926570+0000","last_fullsized":"2023-10-16T08:38:46.926570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.632862+0000","last_change":"2023-10-16T08:38:44.658819+0000","last_active":"2023-10-16T08:38:46.632862+0000","last_peered":"2023-10-16T08:38:46.632862+0000","last_clean":"2023-10-16T08:38:46.632862+0000","last_became_active":"2023-10-16T08:38:44.658257+0000","last_became_peered":"2023-10-16T08:38:44.658257+0000","last_unstale":"2023-10-16T08:38:46.632862+0000","last_undegraded":"2023-10-16T08:38:46.632862+0000","last_fullsized":"2023-10-16T08:38:46.632862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:54.848445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628283+0000","last_change":"2023-10-16T08:38:44.647536+0000","last_active":"2023-10-16T08:38:46.628283+0000","last_peered":"2023-10-16T08:38:46.628283+0000","last_clean":"2023-10-16T08:38:46.628283+0000","last_became_active":"2023-10-16T08:38:44.645931+0000","last_became_peered":"2023-10-16T08:38:44.645931+0000","last_unstale":"2023-10-16T08:38:46.628283+0000","last_undegraded":"2023-10-16T08:38:46.628283+0000","last_fullsized":"2023-10-16T08:38:46.628283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628223+0000","last_change":"2023-10-16T08:38:45.662855+0000","last_active":"2023-10-16T08:38:46.628223+0000","last_peered":"2023-10-16T08:38:46.628223+0000","last_clean":"2023-10-16T08:38:46.628223+0000","last_became_active":"2023-10-16T08:38:45.662072+0000","last_became_peered":"2023-10-16T08:38:45.662072+0000","last_unstale":"2023-10-16T08:38:46.628223+0000","last_undegraded":"2023-10-16T08:38:46.628223+0000","last_fullsized":"2023-10-16T08:38:46.628223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.929953+0000","last_change":"2023-10-16T08:38:44.657651+0000","last_active":"2023-10-16T08:38:46.929953+0000","last_peered":"2023-10-16T08:38:46.929953+0000","last_clean":"2023-10-16T08:38:46.929953+0000","last_became_active":"2023-10-16T08:38:44.657065+0000","last_became_peered":"2023-10-16T08:38:44.657065+0000","last_unstale":"2023-10-16T08:38:46.929953+0000","last_undegraded":"2023-10-16T08:38:46.929953+0000","last_fullsized":"2023-10-16T08:38:46.929953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.625975+0000","last_change":"2023-10-16T08:38:45.668048+0000","last_active":"2023-10-16T08:38:46.625975+0000","last_peered":"2023-10-16T08:38:46.625975+0000","last_clean":"2023-10-16T08:38:46.625975+0000","last_became_active":"2023-10-16T08:38:45.667413+0000","last_became_peered":"2023-10-16T08:38:45.667413+0000","last_unstale":"2023-10-16T08:38:46.625975+0000","last_undegraded":"2023-10-16T08:38:46.625975+0000","last_fullsized":"2023-10-16T08:38:46.625975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633561+0000","last_change":"2023-10-16T08:38:44.663188+0000","last_active":"2023-10-16T08:38:46.633561+0000","last_peered":"2023-10-16T08:38:46.633561+0000","last_clean":"2023-10-16T08:38:46.633561+0000","last_became_active":"2023-10-16T08:38:44.662383+0000","last_became_peered":"2023-10-16T08:38:44.662383+0000","last_unstale":"2023-10-16T08:38:46.633561+0000","last_undegraded":"2023-10-16T08:38:46.633561+0000","last_fullsized":"2023-10-16T08:38:46.633561+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:24.375366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930913+0000","last_change":"2023-10-16T08:38:45.679271+0000","last_active":"2023-10-16T08:38:46.930913+0000","last_peered":"2023-10-16T08:38:46.930913+0000","last_clean":"2023-10-16T08:38:46.930913+0000","last_became_active":"2023-10-16T08:38:45.678050+0000","last_became_peered":"2023-10-16T08:38:45.678050+0000","last_unstale":"2023-10-16T08:38:46.930913+0000","last_undegraded":"2023-10-16T08:38:46.930913+0000","last_fullsized":"2023-10-16T08:38:46.930913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931150+0000","last_change":"2023-10-16T08:38:44.653478+0000","last_active":"2023-10-16T08:38:46.931150+0000","last_peered":"2023-10-16T08:38:46.931150+0000","last_clean":"2023-10-16T08:38:46.931150+0000","last_became_active":"2023-10-16T08:38:44.652618+0000","last_became_peered":"2023-10-16T08:38:44.652618+0000","last_unstale":"2023-10-16T08:38:46.931150+0000","last_undegraded":"2023-10-16T08:38:46.931150+0000","last_fullsized":"2023-10-16T08:38:46.931150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926242+0000","last_change":"2023-10-16T08:38:45.656603+0000","last_active":"2023-10-16T08:38:46.926242+0000","last_peered":"2023-10-16T08:38:46.926242+0000","last_clean":"2023-10-16T08:38:46.926242+0000","last_became_active":"2023-10-16T08:38:45.655213+0000","last_became_peered":"2023-10-16T08:38:45.655213+0000","last_unstale":"2023-10-16T08:38:46.926242+0000","last_undegraded":"2023-10-16T08:38:46.926242+0000","last_fullsized":"2023-10-16T08:38:46.926242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626622+0000","last_change":"2023-10-16T08:38:44.651637+0000","last_active":"2023-10-16T08:38:46.626622+0000","last_peered":"2023-10-16T08:38:46.626622+0000","last_clean":"2023-10-16T08:38:46.626622+0000","last_became_active":"2023-10-16T08:38:44.651359+0000","last_became_peered":"2023-10-16T08:38:44.651359+0000","last_unstale":"2023-10-16T08:38:46.626622+0000","last_undegraded":"2023-10-16T08:38:46.626622+0000","last_fullsized":"2023-10-16T08:38:46.626622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:10.859316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.932633+0000","last_change":"2023-10-16T08:38:45.661741+0000","last_active":"2023-10-16T08:38:46.932633+0000","last_peered":"2023-10-16T08:38:46.932633+0000","last_clean":"2023-10-16T08:38:46.932633+0000","last_became_active":"2023-10-16T08:38:45.660611+0000","last_became_peered":"2023-10-16T08:38:45.660611+0000","last_unstale":"2023-10-16T08:38:46.932633+0000","last_undegraded":"2023-10-16T08:38:46.932633+0000","last_fullsized":"2023-10-16T08:38:46.932633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:55.859300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633359+0000","last_change":"2023-10-16T08:38:44.647593+0000","last_active":"2023-10-16T08:38:46.633359+0000","last_peered":"2023-10-16T08:38:46.633359+0000","last_clean":"2023-10-16T08:38:46.633359+0000","last_became_active":"2023-10-16T08:38:44.647001+0000","last_became_peered":"2023-10-16T08:38:44.647001+0000","last_unstale":"2023-10-16T08:38:46.633359+0000","last_undegraded":"2023-10-16T08:38:46.633359+0000","last_fullsized":"2023-10-16T08:38:46.633359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930291+0000","last_change":"2023-10-16T08:38:45.668629+0000","last_active":"2023-10-16T08:38:46.930291+0000","last_peered":"2023-10-16T08:38:46.930291+0000","last_clean":"2023-10-16T08:38:46.930291+0000","last_became_active":"2023-10-16T08:38:45.667334+0000","last_became_peered":"2023-10-16T08:38:45.667334+0000","last_unstale":"2023-10-16T08:38:46.930291+0000","last_undegraded":"2023-10-16T08:38:46.930291+0000","last_fullsized":"2023-10-16T08:38:46.930291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926736+0000","last_change":"2023-10-16T08:38:44.655832+0000","last_active":"2023-10-16T08:38:46.926736+0000","last_peered":"2023-10-16T08:38:46.926736+0000","last_clean":"2023-10-16T08:38:46.926736+0000","last_became_active":"2023-10-16T08:38:44.655469+0000","last_became_peered":"2023-10-16T08:38:44.655469+0000","last_unstale":"2023-10-16T08:38:46.926736+0000","last_undegraded":"2023-10-16T08:38:46.926736+0000","last_fullsized":"2023-10-16T08:38:46.926736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:19.768762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926112+0000","last_change":"2023-10-16T08:38:45.653688+0000","last_active":"2023-10-16T08:38:46.926112+0000","last_peered":"2023-10-16T08:38:46.926112+0000","last_clean":"2023-10-16T08:38:46.926112+0000","last_became_active":"2023-10-16T08:38:45.650563+0000","last_became_peered":"2023-10-16T08:38:45.650563+0000","last_unstale":"2023-10-16T08:38:46.926112+0000","last_undegraded":"2023-10-16T08:38:46.926112+0000","last_fullsized":"2023-10-16T08:38:46.926112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:29.539285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931330+0000","last_change":"2023-10-16T08:38:44.662816+0000","last_active":"2023-10-16T08:38:46.931330+0000","last_peered":"2023-10-16T08:38:46.931330+0000","last_clean":"2023-10-16T08:38:46.931330+0000","last_became_active":"2023-10-16T08:38:44.662110+0000","last_became_peered":"2023-10-16T08:38:44.662110+0000","last_unstale":"2023-10-16T08:38:46.931330+0000","last_undegraded":"2023-10-16T08:38:46.931330+0000","last_fullsized":"2023-10-16T08:38:46.931330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:00.201724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930788+0000","last_change":"2023-10-16T08:38:45.662166+0000","last_active":"2023-10-16T08:38:46.930788+0000","last_peered":"2023-10-16T08:38:46.930788+0000","last_clean":"2023-10-16T08:38:46.930788+0000","last_became_active":"2023-10-16T08:38:45.661595+0000","last_became_peered":"2023-10-16T08:38:45.661595+0000","last_unstale":"2023-10-16T08:38:46.930788+0000","last_undegraded":"2023-10-16T08:38:46.930788+0000","last_fullsized":"2023-10-16T08:38:46.930788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:02:39.840175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.632731+0000","last_change":"2023-10-16T08:38:44.657766+0000","last_active":"2023-10-16T08:38:46.632731+0000","last_peered":"2023-10-16T08:38:46.632731+0000","last_clean":"2023-10-16T08:38:46.632731+0000","last_became_active":"2023-10-16T08:38:44.657153+0000","last_became_peered":"2023-10-16T08:38:44.657153+0000","last_unstale":"2023-10-16T08:38:46.632731+0000","last_undegraded":"2023-10-16T08:38:46.632731+0000","last_fullsized":"2023-10-16T08:38:46.632731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.632435+0000","last_change":"2023-10-16T08:38:45.637591+0000","last_active":"2023-10-16T08:38:46.632435+0000","last_peered":"2023-10-16T08:38:46.632435+0000","last_clean":"2023-10-16T08:38:46.632435+0000","last_became_active":"2023-10-16T08:38:45.635633+0000","last_became_peered":"2023-10-16T08:38:45.635633+0000","last_unstale":"2023-10-16T08:38:46.632435+0000","last_undegraded":"2023-10-16T08:38:46.632435+0000","last_fullsized":"2023-10-16T08:38:46.632435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927363+0000","last_change":"2023-10-16T08:38:44.638297+0000","last_active":"2023-10-16T08:38:46.927363+0000","last_peered":"2023-10-16T08:38:46.927363+0000","last_clean":"2023-10-16T08:38:46.927363+0000","last_became_active":"2023-10-16T08:38:44.637640+0000","last_became_peered":"2023-10-16T08:38:44.637640+0000","last_unstale":"2023-10-16T08:38:46.927363+0000","last_undegraded":"2023-10-16T08:38:46.927363+0000","last_fullsized":"2023-10-16T08:38:46.927363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:16.003943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931498+0000","last_change":"2023-10-16T08:38:45.657286+0000","last_active":"2023-10-16T08:38:46.931498+0000","last_peered":"2023-10-16T08:38:46.931498+0000","last_clean":"2023-10-16T08:38:46.931498+0000","last_became_active":"2023-10-16T08:38:45.656651+0000","last_became_peered":"2023-10-16T08:38:45.656651+0000","last_unstale":"2023-10-16T08:38:46.931498+0000","last_undegraded":"2023-10-16T08:38:46.931498+0000","last_fullsized":"2023-10-16T08:38:46.931498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:28.052483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.932737+0000","last_change":"2023-10-16T08:38:44.648434+0000","last_active":"2023-10-16T08:38:46.932737+0000","last_peered":"2023-10-16T08:38:46.932737+0000","last_clean":"2023-10-16T08:38:46.932737+0000","last_became_active":"2023-10-16T08:38:44.647887+0000","last_became_peered":"2023-10-16T08:38:44.647887+0000","last_unstale":"2023-10-16T08:38:46.932737+0000","last_undegraded":"2023-10-16T08:38:46.932737+0000","last_fullsized":"2023-10-16T08:38:46.932737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930414+0000","last_change":"2023-10-16T08:38:45.661965+0000","last_active":"2023-10-16T08:38:46.930414+0000","last_peered":"2023-10-16T08:38:46.930414+0000","last_clean":"2023-10-16T08:38:46.930414+0000","last_became_active":"2023-10-16T08:38:45.656750+0000","last_became_peered":"2023-10-16T08:38:45.656750+0000","last_unstale":"2023-10-16T08:38:46.930414+0000","last_undegraded":"2023-10-16T08:38:46.930414+0000","last_fullsized":"2023-10-16T08:38:46.930414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927746+0000","last_change":"2023-10-16T08:38:44.647524+0000","last_active":"2023-10-16T08:38:46.927746+0000","last_peered":"2023-10-16T08:38:46.927746+0000","last_clean":"2023-10-16T08:38:46.927746+0000","last_became_active":"2023-10-16T08:38:44.646042+0000","last_became_peered":"2023-10-16T08:38:44.646042+0000","last_unstale":"2023-10-16T08:38:46.927746+0000","last_undegraded":"2023-10-16T08:38:46.927746+0000","last_fullsized":"2023-10-16T08:38:46.927746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:24.375366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931909+0000","last_change":"2023-10-16T08:38:45.670969+0000","last_active":"2023-10-16T08:38:46.931909+0000","last_peered":"2023-10-16T08:38:46.931909+0000","last_clean":"2023-10-16T08:38:46.931909+0000","last_became_active":"2023-10-16T08:38:45.670415+0000","last_became_peered":"2023-10-16T08:38:45.670415+0000","last_unstale":"2023-10-16T08:38:46.931909+0000","last_undegraded":"2023-10-16T08:38:46.931909+0000","last_fullsized":"2023-10-16T08:38:46.931909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:15:00.472153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626608+0000","last_change":"2023-10-16T08:38:44.655744+0000","last_active":"2023-10-16T08:38:46.626608+0000","last_peered":"2023-10-16T08:38:46.626608+0000","last_clean":"2023-10-16T08:38:46.626608+0000","last_became_active":"2023-10-16T08:38:44.655260+0000","last_became_peered":"2023-10-16T08:38:44.655260+0000","last_unstale":"2023-10-16T08:38:46.626608+0000","last_undegraded":"2023-10-16T08:38:46.626608+0000","last_fullsized":"2023-10-16T08:38:46.626608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:54.848445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931710+0000","last_change":"2023-10-16T08:38:45.668230+0000","last_active":"2023-10-16T08:38:46.931710+0000","last_peered":"2023-10-16T08:38:46.931710+0000","last_clean":"2023-10-16T08:38:46.931710+0000","last_became_active":"2023-10-16T08:38:45.667627+0000","last_became_peered":"2023-10-16T08:38:45.667627+0000","last_unstale":"2023-10-16T08:38:46.931710+0000","last_undegraded":"2023-10-16T08:38:46.931710+0000","last_fullsized":"2023-10-16T08:38:46.931710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:28.052483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927508+0000","last_change":"2023-10-16T08:38:44.662118+0000","last_active":"2023-10-16T08:38:46.927508+0000","last_peered":"2023-10-16T08:38:46.927508+0000","last_clean":"2023-10-16T08:38:46.927508+0000","last_became_active":"2023-10-16T08:38:44.661736+0000","last_became_peered":"2023-10-16T08:38:44.661736+0000","last_unstale":"2023-10-16T08:38:46.927508+0000","last_undegraded":"2023-10-16T08:38:46.927508+0000","last_fullsized":"2023-10-16T08:38:46.927508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:49:40.967349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930619+0000","last_change":"2023-10-16T08:38:45.667423+0000","last_active":"2023-10-16T08:38:46.930619+0000","last_peered":"2023-10-16T08:38:46.930619+0000","last_clean":"2023-10-16T08:38:46.930619+0000","last_became_active":"2023-10-16T08:38:45.666390+0000","last_became_peered":"2023-10-16T08:38:45.666390+0000","last_unstale":"2023-10-16T08:38:46.930619+0000","last_undegraded":"2023-10-16T08:38:46.930619+0000","last_fullsized":"2023-10-16T08:38:46.930619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:33.190218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-16T08:38:45.628438+0000","last_change":"2023-10-16T08:38:34.313511+0000","last_active":"2023-10-16T08:38:45.628438+0000","last_peered":"2023-10-16T08:38:45.628438+0000","last_clean":"2023-10-16T08:38:45.628438+0000","last_became_active":"2023-10-16T08:38:34.310944+0000","last_became_peered":"2023-10-16T08:38:34.310944+0000","last_unstale":"2023-10-16T08:38:45.628438+0000","last_undegraded":"2023-10-16T08:38:45.628438+0000","last_fullsized":"2023-10-16T08:38:45.628438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:33.264692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:33.264692+0000","last_clean_scrub_stamp":"2023-10-16T08:38:33.264692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:50.170461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930271+0000","last_change":"2023-10-16T08:38:44.655223+0000","last_active":"2023-10-16T08:38:46.930271+0000","last_peered":"2023-10-16T08:38:46.930271+0000","last_clean":"2023-10-16T08:38:46.930271+0000","last_became_active":"2023-10-16T08:38:44.654277+0000","last_became_peered":"2023-10-16T08:38:44.654277+0000","last_unstale":"2023-10-16T08:38:46.930271+0000","last_undegraded":"2023-10-16T08:38:46.930271+0000","last_fullsized":"2023-10-16T08:38:46.930271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:03.462433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628392+0000","last_change":"2023-10-16T08:38:45.679868+0000","last_active":"2023-10-16T08:38:46.628392+0000","last_peered":"2023-10-16T08:38:46.628392+0000","last_clean":"2023-10-16T08:38:46.628392+0000","last_became_active":"2023-10-16T08:38:45.679409+0000","last_became_peered":"2023-10-16T08:38:45.679409+0000","last_unstale":"2023-10-16T08:38:46.628392+0000","last_undegraded":"2023-10-16T08:38:46.628392+0000","last_fullsized":"2023-10-16T08:38:46.628392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930490+0000","last_change":"2023-10-16T08:38:44.652152+0000","last_active":"2023-10-16T08:38:46.930490+0000","last_peered":"2023-10-16T08:38:46.930490+0000","last_clean":"2023-10-16T08:38:46.930490+0000","last_became_active":"2023-10-16T08:38:44.651534+0000","last_became_peered":"2023-10-16T08:38:44.651534+0000","last_unstale":"2023-10-16T08:38:46.930490+0000","last_undegraded":"2023-10-16T08:38:46.930490+0000","last_fullsized":"2023-10-16T08:38:46.930490+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926115+0000","last_change":"2023-10-16T08:38:45.651786+0000","last_active":"2023-10-16T08:38:46.926115+0000","last_peered":"2023-10-16T08:38:46.926115+0000","last_clean":"2023-10-16T08:38:46.926115+0000","last_became_active":"2023-10-16T08:38:45.650715+0000","last_became_peered":"2023-10-16T08:38:45.650715+0000","last_unstale":"2023-10-16T08:38:46.926115+0000","last_undegraded":"2023-10-16T08:38:46.926115+0000","last_fullsized":"2023-10-16T08:38:46.926115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:29.799799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927326+0000","last_change":"2023-10-16T08:38:44.645933+0000","last_active":"2023-10-16T08:38:46.927326+0000","last_peered":"2023-10-16T08:38:46.927326+0000","last_clean":"2023-10-16T08:38:46.927326+0000","last_became_active":"2023-10-16T08:38:44.645451+0000","last_became_peered":"2023-10-16T08:38:44.645451+0000","last_unstale":"2023-10-16T08:38:46.927326+0000","last_undegraded":"2023-10-16T08:38:46.927326+0000","last_fullsized":"2023-10-16T08:38:46.927326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.625977+0000","last_change":"2023-10-16T08:38:45.641499+0000","last_active":"2023-10-16T08:38:46.625977+0000","last_peered":"2023-10-16T08:38:46.625977+0000","last_clean":"2023-10-16T08:38:46.625977+0000","last_became_active":"2023-10-16T08:38:45.640765+0000","last_became_peered":"2023-10-16T08:38:45.640765+0000","last_unstale":"2023-10-16T08:38:46.625977+0000","last_undegraded":"2023-10-16T08:38:46.625977+0000","last_fullsized":"2023-10-16T08:38:46.625977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931417+0000","last_change":"2023-10-16T08:38:44.655401+0000","last_active":"2023-10-16T08:38:46.931417+0000","last_peered":"2023-10-16T08:38:46.931417+0000","last_clean":"2023-10-16T08:38:46.931417+0000","last_became_active":"2023-10-16T08:38:44.654622+0000","last_became_peered":"2023-10-16T08:38:44.654622+0000","last_unstale":"2023-10-16T08:38:46.931417+0000","last_undegraded":"2023-10-16T08:38:46.931417+0000","last_fullsized":"2023-10-16T08:38:46.931417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:48.705915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.932454+0000","last_change":"2023-10-16T08:38:45.659377+0000","last_active":"2023-10-16T08:38:46.932454+0000","last_peered":"2023-10-16T08:38:46.932454+0000","last_clean":"2023-10-16T08:38:46.932454+0000","last_became_active":"2023-10-16T08:38:45.658594+0000","last_became_peered":"2023-10-16T08:38:45.658594+0000","last_unstale":"2023-10-16T08:38:46.932454+0000","last_undegraded":"2023-10-16T08:38:46.932454+0000","last_fullsized":"2023-10-16T08:38:46.932454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:56.738058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626155+0000","last_change":"2023-10-16T08:38:44.634158+0000","last_active":"2023-10-16T08:38:46.626155+0000","last_peered":"2023-10-16T08:38:46.626155+0000","last_clean":"2023-10-16T08:38:46.626155+0000","last_became_active":"2023-10-16T08:38:44.633854+0000","last_became_peered":"2023-10-16T08:38:44.633854+0000","last_unstale":"2023-10-16T08:38:46.626155+0000","last_undegraded":"2023-10-16T08:38:46.626155+0000","last_fullsized":"2023-10-16T08:38:46.626155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:55.727203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931328+0000","last_change":"2023-10-16T08:38:45.675931+0000","last_active":"2023-10-16T08:38:46.931328+0000","last_peered":"2023-10-16T08:38:46.931328+0000","last_clean":"2023-10-16T08:38:46.931328+0000","last_became_active":"2023-10-16T08:38:45.674962+0000","last_became_peered":"2023-10-16T08:38:45.674962+0000","last_unstale":"2023-10-16T08:38:46.931328+0000","last_undegraded":"2023-10-16T08:38:46.931328+0000","last_fullsized":"2023-10-16T08:38:46.931328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:13.453321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931397+0000","last_change":"2023-10-16T08:38:44.655693+0000","last_active":"2023-10-16T08:38:46.931397+0000","last_peered":"2023-10-16T08:38:46.931397+0000","last_clean":"2023-10-16T08:38:46.931397+0000","last_became_active":"2023-10-16T08:38:44.655001+0000","last_became_peered":"2023-10-16T08:38:44.655001+0000","last_unstale":"2023-10-16T08:38:46.931397+0000","last_undegraded":"2023-10-16T08:38:46.931397+0000","last_fullsized":"2023-10-16T08:38:46.931397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:30.649639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626084+0000","last_change":"2023-10-16T08:38:45.639658+0000","last_active":"2023-10-16T08:38:46.626084+0000","last_peered":"2023-10-16T08:38:46.626084+0000","last_clean":"2023-10-16T08:38:46.626084+0000","last_became_active":"2023-10-16T08:38:45.638702+0000","last_became_peered":"2023-10-16T08:38:45.638702+0000","last_unstale":"2023-10-16T08:38:46.626084+0000","last_undegraded":"2023-10-16T08:38:46.626084+0000","last_fullsized":"2023-10-16T08:38:46.626084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:47.979511+0000","last_change":"2023-10-16T08:38:44.648715+0000","last_active":"2023-10-16T08:38:47.979511+0000","last_peered":"2023-10-16T08:38:47.979511+0000","last_clean":"2023-10-16T08:38:47.979511+0000","last_became_active":"2023-10-16T08:38:44.648312+0000","last_became_peered":"2023-10-16T08:38:44.648312+0000","last_unstale":"2023-10-16T08:38:47.979511+0000","last_undegraded":"2023-10-16T08:38:47.979511+0000","last_fullsized":"2023-10-16T08:38:47.979511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628807+0000","last_change":"2023-10-16T08:38:45.663402+0000","last_active":"2023-10-16T08:38:46.628807+0000","last_peered":"2023-10-16T08:38:46.628807+0000","last_clean":"2023-10-16T08:38:46.628807+0000","last_became_active":"2023-10-16T08:38:45.662442+0000","last_became_peered":"2023-10-16T08:38:45.662442+0000","last_unstale":"2023-10-16T08:38:46.628807+0000","last_undegraded":"2023-10-16T08:38:46.628807+0000","last_fullsized":"2023-10-16T08:38:46.628807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633639+0000","last_change":"2023-10-16T08:38:44.650015+0000","last_active":"2023-10-16T08:38:46.633639+0000","last_peered":"2023-10-16T08:38:46.633639+0000","last_clean":"2023-10-16T08:38:46.633639+0000","last_became_active":"2023-10-16T08:38:44.648155+0000","last_became_peered":"2023-10-16T08:38:44.648155+0000","last_unstale":"2023-10-16T08:38:46.633639+0000","last_undegraded":"2023-10-16T08:38:46.633639+0000","last_fullsized":"2023-10-16T08:38:46.633639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926726+0000","last_change":"2023-10-16T08:38:45.637568+0000","last_active":"2023-10-16T08:38:46.926726+0000","last_peered":"2023-10-16T08:38:46.926726+0000","last_clean":"2023-10-16T08:38:46.926726+0000","last_became_active":"2023-10-16T08:38:45.636751+0000","last_became_peered":"2023-10-16T08:38:45.636751+0000","last_unstale":"2023-10-16T08:38:46.926726+0000","last_undegraded":"2023-10-16T08:38:46.926726+0000","last_fullsized":"2023-10-16T08:38:46.926726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930247+0000","last_change":"2023-10-16T08:38:44.649257+0000","last_active":"2023-10-16T08:38:46.930247+0000","last_peered":"2023-10-16T08:38:46.930247+0000","last_clean":"2023-10-16T08:38:46.930247+0000","last_became_active":"2023-10-16T08:38:44.646990+0000","last_became_peered":"2023-10-16T08:38:44.646990+0000","last_unstale":"2023-10-16T08:38:46.930247+0000","last_undegraded":"2023-10-16T08:38:46.930247+0000","last_fullsized":"2023-10-16T08:38:46.930247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930389+0000","last_change":"2023-10-16T08:38:45.648228+0000","last_active":"2023-10-16T08:38:46.930389+0000","last_peered":"2023-10-16T08:38:46.930389+0000","last_clean":"2023-10-16T08:38:46.930389+0000","last_became_active":"2023-10-16T08:38:45.647449+0000","last_became_peered":"2023-10-16T08:38:45.647449+0000","last_unstale":"2023-10-16T08:38:46.930389+0000","last_undegraded":"2023-10-16T08:38:46.930389+0000","last_fullsized":"2023-10-16T08:38:46.930389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:20.779617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626045+0000","last_change":"2023-10-16T08:38:44.631354+0000","last_active":"2023-10-16T08:38:46.626045+0000","last_peered":"2023-10-16T08:38:46.626045+0000","last_clean":"2023-10-16T08:38:46.626045+0000","last_became_active":"2023-10-16T08:38:44.630451+0000","last_became_peered":"2023-10-16T08:38:44.630451+0000","last_unstale":"2023-10-16T08:38:46.626045+0000","last_undegraded":"2023-10-16T08:38:46.626045+0000","last_fullsized":"2023-10-16T08:38:46.626045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:10.980982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628323+0000","last_change":"2023-10-16T08:38:45.663898+0000","last_active":"2023-10-16T08:38:46.628323+0000","last_peered":"2023-10-16T08:38:46.628323+0000","last_clean":"2023-10-16T08:38:46.628323+0000","last_became_active":"2023-10-16T08:38:45.663100+0000","last_became_peered":"2023-10-16T08:38:45.663100+0000","last_unstale":"2023-10-16T08:38:46.628323+0000","last_undegraded":"2023-10-16T08:38:46.628323+0000","last_fullsized":"2023-10-16T08:38:46.628323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626257+0000","last_change":"2023-10-16T08:38:44.646987+0000","last_active":"2023-10-16T08:38:46.626257+0000","last_peered":"2023-10-16T08:38:46.626257+0000","last_clean":"2023-10-16T08:38:46.626257+0000","last_became_active":"2023-10-16T08:38:44.646580+0000","last_became_peered":"2023-10-16T08:38:44.646580+0000","last_unstale":"2023-10-16T08:38:46.626257+0000","last_undegraded":"2023-10-16T08:38:46.626257+0000","last_fullsized":"2023-10-16T08:38:46.626257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628448+0000","last_change":"2023-10-16T08:38:45.662894+0000","last_active":"2023-10-16T08:38:46.628448+0000","last_peered":"2023-10-16T08:38:46.628448+0000","last_clean":"2023-10-16T08:38:46.628448+0000","last_became_active":"2023-10-16T08:38:45.662226+0000","last_became_peered":"2023-10-16T08:38:45.662226+0000","last_unstale":"2023-10-16T08:38:46.628448+0000","last_undegraded":"2023-10-16T08:38:46.628448+0000","last_fullsized":"2023-10-16T08:38:46.628448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931043+0000","last_change":"2023-10-16T08:38:44.660571+0000","last_active":"2023-10-16T08:38:46.931043+0000","last_peered":"2023-10-16T08:38:46.931043+0000","last_clean":"2023-10-16T08:38:46.931043+0000","last_became_active":"2023-10-16T08:38:44.659976+0000","last_became_peered":"2023-10-16T08:38:44.659976+0000","last_unstale":"2023-10-16T08:38:46.931043+0000","last_undegraded":"2023-10-16T08:38:46.931043+0000","last_fullsized":"2023-10-16T08:38:46.931043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.632966+0000","last_change":"2023-10-16T08:38:45.665200+0000","last_active":"2023-10-16T08:38:46.632966+0000","last_peered":"2023-10-16T08:38:46.632966+0000","last_clean":"2023-10-16T08:38:46.632966+0000","last_became_active":"2023-10-16T08:38:45.664616+0000","last_became_peered":"2023-10-16T08:38:45.664616+0000","last_unstale":"2023-10-16T08:38:46.632966+0000","last_undegraded":"2023-10-16T08:38:46.632966+0000","last_fullsized":"2023-10-16T08:38:46.632966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.933149+0000","last_change":"2023-10-16T08:38:45.680892+0000","last_active":"2023-10-16T08:38:46.933149+0000","last_peered":"2023-10-16T08:38:46.933149+0000","last_clean":"2023-10-16T08:38:46.933149+0000","last_became_active":"2023-10-16T08:38:45.680231+0000","last_became_peered":"2023-10-16T08:38:45.680231+0000","last_unstale":"2023-10-16T08:38:46.933149+0000","last_undegraded":"2023-10-16T08:38:46.933149+0000","last_fullsized":"2023-10-16T08:38:46.933149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:58:51.599709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.929904+0000","last_change":"2023-10-16T08:38:44.664269+0000","last_active":"2023-10-16T08:38:46.929904+0000","last_peered":"2023-10-16T08:38:46.929904+0000","last_clean":"2023-10-16T08:38:46.929904+0000","last_became_active":"2023-10-16T08:38:44.663674+0000","last_became_peered":"2023-10-16T08:38:44.663674+0000","last_unstale":"2023-10-16T08:38:46.929904+0000","last_undegraded":"2023-10-16T08:38:46.929904+0000","last_fullsized":"2023-10-16T08:38:46.929904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931372+0000","last_change":"2023-10-16T08:38:45.680890+0000","last_active":"2023-10-16T08:38:46.931372+0000","last_peered":"2023-10-16T08:38:46.931372+0000","last_clean":"2023-10-16T08:38:46.931372+0000","last_became_active":"2023-10-16T08:38:45.680266+0000","last_became_peered":"2023-10-16T08:38:45.680266+0000","last_unstale":"2023-10-16T08:38:46.931372+0000","last_undegraded":"2023-10-16T08:38:46.931372+0000","last_fullsized":"2023-10-16T08:38:46.931372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:26.292137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930850+0000","last_change":"2023-10-16T08:38:44.653447+0000","last_active":"2023-10-16T08:38:46.930850+0000","last_peered":"2023-10-16T08:38:46.930850+0000","last_clean":"2023-10-16T08:38:46.930850+0000","last_became_active":"2023-10-16T08:38:44.652658+0000","last_became_peered":"2023-10-16T08:38:44.652658+0000","last_unstale":"2023-10-16T08:38:46.930850+0000","last_undegraded":"2023-10-16T08:38:46.930850+0000","last_fullsized":"2023-10-16T08:38:46.930850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:22.350066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926151+0000","last_change":"2023-10-16T08:38:45.665041+0000","last_active":"2023-10-16T08:38:46.926151+0000","last_peered":"2023-10-16T08:38:46.926151+0000","last_clean":"2023-10-16T08:38:46.926151+0000","last_became_active":"2023-10-16T08:38:45.664274+0000","last_became_peered":"2023-10-16T08:38:45.664274+0000","last_unstale":"2023-10-16T08:38:46.926151+0000","last_undegraded":"2023-10-16T08:38:46.926151+0000","last_fullsized":"2023-10-16T08:38:46.926151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628731+0000","last_change":"2023-10-16T08:38:44.645754+0000","last_active":"2023-10-16T08:38:46.628731+0000","last_peered":"2023-10-16T08:38:46.628731+0000","last_clean":"2023-10-16T08:38:46.628731+0000","last_became_active":"2023-10-16T08:38:44.645000+0000","last_became_peered":"2023-10-16T08:38:44.645000+0000","last_unstale":"2023-10-16T08:38:46.628731+0000","last_undegraded":"2023-10-16T08:38:46.628731+0000","last_fullsized":"2023-10-16T08:38:46.628731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628628+0000","last_change":"2023-10-16T08:38:45.664252+0000","last_active":"2023-10-16T08:38:46.628628+0000","last_peered":"2023-10-16T08:38:46.628628+0000","last_clean":"2023-10-16T08:38:46.628628+0000","last_became_active":"2023-10-16T08:38:45.663538+0000","last_became_peered":"2023-10-16T08:38:45.663538+0000","last_unstale":"2023-10-16T08:38:46.628628+0000","last_undegraded":"2023-10-16T08:38:46.628628+0000","last_fullsized":"2023-10-16T08:38:46.628628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926310+0000","last_change":"2023-10-16T08:38:44.646321+0000","last_active":"2023-10-16T08:38:46.926310+0000","last_peered":"2023-10-16T08:38:46.926310+0000","last_clean":"2023-10-16T08:38:46.926310+0000","last_became_active":"2023-10-16T08:38:44.645854+0000","last_became_peered":"2023-10-16T08:38:44.645854+0000","last_unstale":"2023-10-16T08:38:46.926310+0000","last_undegraded":"2023-10-16T08:38:46.926310+0000","last_fullsized":"2023-10-16T08:38:46.926310+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:10.980982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":70988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.534}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:38:53.310 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:38:53.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.460+0000 7f0f60d53640 1 Processor -- start 2023-10-16T08:38:53.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.464+0000 7f0f60d53640 1 -- start start 2023-10-16T08:38:53.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.464+0000 7f0f60d53640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f5c15b2e0 0x7f0f5c0cc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:53.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.464+0000 7f0f60d53640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0f5c0ccd70 0x7f0f5c0cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:53.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.464+0000 7f0f60d53640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f5c06fc20 con 0x7f0f5c0af6d0 2023-10-16T08:38:53.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.464+0000 7f0f60d53640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f5c0cb380 con 0x7f0f5c0ccd70 2023-10-16T08:38:53.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.464+0000 7f0f60d53640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c0b3200 con 0x7f0f5c15b2e0 2023-10-16T08:38:53.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.464+0000 7f0f5a59c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f5c15b2e0 0x7f0f5c0cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:53.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.464+0000 7f0f59d9b640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0f5c0ccd70 0x7f0f5c0cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:53.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.464+0000 7f0f5ad9d640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0f5c0af6d0 0x7f0f5c0b2480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.40:47302/0 (socket says 172.21.15.40:47302) 2023-10-16T08:38:53.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.464+0000 7f0f5ad9d640 1 -- 172.21.15.40:0/2625958982 learned_addr learned my addr 172.21.15.40:0/2625958982 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:53.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.464+0000 7f0f5a59c640 1 -- 172.21.15.40:0/2625958982 >> v2:172.21.15.86:3300/0 conn(0x7f0f5c0ccd70 msgr2=0x7f0f5c0cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:53.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.464+0000 7f0f5a59c640 1 --2- 172.21.15.40:0/2625958982 >> v2:172.21.15.86:3300/0 conn(0x7f0f5c0ccd70 0x7f0f5c0cf310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.464+0000 7f0f5a59c640 1 -- 172.21.15.40:0/2625958982 >> v1:172.21.15.86:6789/0 conn(0x7f0f5c0af6d0 legacy=0x7f0f5c0b2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:53.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.464+0000 7f0f5a59c640 1 -- 172.21.15.40:0/2625958982 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f5c06fee0 con 0x7f0f5c15b2e0 2023-10-16T08:38:53.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f5a59c640 1 --2- 172.21.15.40:0/2625958982 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f5c15b2e0 0x7f0f5c0cc830 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0f500097c0 tx=0x7f0f5001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa50c9369253c925 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:53.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f5959a640 1 -- 172.21.15.40:0/2625958982 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f500ab680 con 0x7f0f5c15b2e0 2023-10-16T08:38:53.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f5959a640 1 -- 172.21.15.40:0/2625958982 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f500ab7e0 con 0x7f0f5c15b2e0 2023-10-16T08:38:53.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f5959a640 1 -- 172.21.15.40:0/2625958982 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f500abb70 con 0x7f0f5c15b2e0 2023-10-16T08:38:53.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f60d53640 1 -- 172.21.15.40:0/2625958982 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f5c15b2e0 msgr2=0x7f0f5c0cc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:53.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f60d53640 1 --2- 172.21.15.40:0/2625958982 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f5c15b2e0 0x7f0f5c0cc830 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0f500097c0 tx=0x7f0f5001c480 comp rx=0 tx=0).stop 2023-10-16T08:38:53.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f60d53640 1 -- 172.21.15.40:0/2625958982 shutdown_connections 2023-10-16T08:38:53.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f60d53640 1 --2- 172.21.15.40:0/2625958982 >> v2:172.21.15.86:3300/0 conn(0x7f0f5c0ccd70 0x7f0f5c0cf310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f60d53640 1 --2- 172.21.15.40:0/2625958982 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f5c15b2e0 0x7f0f5c0cc830 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f60d53640 1 -- 172.21.15.40:0/2625958982 >> 172.21.15.40:0/2625958982 conn(0x7f0f5c06d700 msgr2=0x7f0f5c06db10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:53.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f60d53640 1 -- 172.21.15.40:0/2625958982 shutdown_connections 2023-10-16T08:38:53.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f60d53640 1 -- 172.21.15.40:0/2625958982 wait complete. 2023-10-16T08:38:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f60d53640 1 Processor -- start 2023-10-16T08:38:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f60d53640 1 -- start start 2023-10-16T08:38:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f60d53640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f5c0af6d0 0x7f0f5c156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:53.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f60d53640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f5c0ccd70 0x7f0f5c15b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f60d53640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f5c15bc10 0x7f0f5c1652a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f60d53640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c0c8c60 con 0x7f0f5c15bc10 2023-10-16T08:38:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f60d53640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c156970 con 0x7f0f5c0ccd70 2023-10-16T08:38:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f60d53640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f5c15c0b0 con 0x7f0f5c0af6d0 2023-10-16T08:38:53.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f5ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f5c15bc10 0x7f0f5c1652a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f5ad9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f5c15bc10 0x7f0f5c1652a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58522/0 (socket says 172.21.15.40:58522) 2023-10-16T08:38:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f5ad9d640 1 -- 172.21.15.40:0/3002687865 learned_addr learned my addr 172.21.15.40:0/3002687865 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f5a59c640 1 --2- 172.21.15.40:0/3002687865 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f5c0af6d0 0x7f0f5c156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f59d9b640 1 --2- 172.21.15.40:0/3002687865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f5c0ccd70 0x7f0f5c15b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f5ad9d640 1 -- 172.21.15.40:0/3002687865 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f5c0af6d0 msgr2=0x7f0f5c156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:53.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f5ad9d640 1 --2- 172.21.15.40:0/3002687865 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f5c0af6d0 0x7f0f5c156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f5ad9d640 1 -- 172.21.15.40:0/3002687865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f5c0ccd70 msgr2=0x7f0f5c15b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f5ad9d640 1 --2- 172.21.15.40:0/3002687865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f5c0ccd70 0x7f0f5c15b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f5ad9d640 1 -- 172.21.15.40:0/3002687865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f50009420 con 0x7f0f5c15bc10 2023-10-16T08:38:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f5a59c640 1 --2- 172.21.15.40:0/3002687865 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f5c0af6d0 0x7f0f5c156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:38:53.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f59d9b640 1 --2- 172.21.15.40:0/3002687865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f5c0ccd70 0x7f0f5c15b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:38:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.468+0000 7f0f5ad9d640 1 --2- 172.21.15.40:0/3002687865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f5c15bc10 0x7f0f5c1652a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400cbc0 tx=0x7f0f5400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.472+0000 7f0f4affd640 1 -- 172.21.15.40:0/3002687865 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5400ad40 con 0x7f0f5c15bc10 2023-10-16T08:38:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.472+0000 7f0f4affd640 1 -- 172.21.15.40:0/3002687865 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f54012400 con 0x7f0f5c15bc10 2023-10-16T08:38:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.472+0000 7f0f60d53640 1 -- 172.21.15.40:0/3002687865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f5c15c390 con 0x7f0f5c15bc10 2023-10-16T08:38:53.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.472+0000 7f0f4affd640 1 -- 172.21.15.40:0/3002687865 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f5400aea0 con 0x7f0f5c15bc10 2023-10-16T08:38:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.472+0000 7f0f60d53640 1 -- 172.21.15.40:0/3002687865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f5c0c88f0 con 0x7f0f5c15bc10 2023-10-16T08:38:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.472+0000 7f0f60d53640 1 -- 172.21.15.40:0/3002687865 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f5c0c8dd0 con 0x7f0f5c15bc10 2023-10-16T08:38:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.472+0000 7f0f4affd640 1 -- 172.21.15.40:0/3002687865 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0f54007070 con 0x7f0f5c15bc10 2023-10-16T08:38:53.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.472+0000 7f0f4affd640 1 --2- 172.21.15.40:0/3002687865 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0f3c0664a0 0x7f0f3c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.472+0000 7f0f4affd640 1 -- 172.21.15.40:0/3002687865 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f54083cd0 con 0x7f0f5c15bc10 2023-10-16T08:38:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.472+0000 7f0f5a59c640 1 --2- 172.21.15.40:0/3002687865 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0f3c0664a0 0x7f0f3c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:53.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.472+0000 7f0f5a59c640 1 --2- 172.21.15.40:0/3002687865 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0f3c0664a0 0x7f0f3c068960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0f500af000 tx=0x7f0f500a7000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:53.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.476+0000 7f0f4affd640 1 -- 172.21.15.40:0/3002687865 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f0f54051200 con 0x7f0f5c15bc10 2023-10-16T08:38:53.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.628+0000 7f0f60d53640 1 -- 172.21.15.40:0/3002687865 --> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0f5c0b2fa0 con 0x7f0f3c0664a0 2023-10-16T08:38:53.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.648+0000 7f0f4affd640 1 -- 172.21.15.40:0/3002687865 <== mgr.4111 v2:172.21.15.86:6832/1188970899 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264499 (secure 0 0 0) 0x7f0f50006e20 con 0x7f0f3c0664a0 2023-10-16T08:38:53.655 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:53.657 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-16T08:38:53.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.652+0000 7f0f60d53640 1 -- 172.21.15.40:0/3002687865 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0f3c0664a0 msgr2=0x7f0f3c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:53.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.652+0000 7f0f60d53640 1 --2- 172.21.15.40:0/3002687865 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0f3c0664a0 0x7f0f3c068960 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f0f500af000 tx=0x7f0f500a7000 comp rx=0 tx=0).stop 2023-10-16T08:38:53.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.652+0000 7f0f60d53640 1 -- 172.21.15.40:0/3002687865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f5c15bc10 msgr2=0x7f0f5c1652a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:53.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.652+0000 7f0f60d53640 1 --2- 172.21.15.40:0/3002687865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f5c15bc10 0x7f0f5c1652a0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0f5400cbc0 tx=0x7f0f5400a430 comp rx=0 tx=0).stop 2023-10-16T08:38:53.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.652+0000 7f0f60d53640 1 -- 172.21.15.40:0/3002687865 shutdown_connections 2023-10-16T08:38:53.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.652+0000 7f0f60d53640 1 --2- 172.21.15.40:0/3002687865 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0f3c0664a0 0x7f0f3c068960 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.652+0000 7f0f60d53640 1 --2- 172.21.15.40:0/3002687865 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f5c15bc10 0x7f0f5c1652a0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.652+0000 7f0f60d53640 1 --2- 172.21.15.40:0/3002687865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f5c0ccd70 0x7f0f5c15b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.652+0000 7f0f60d53640 1 --2- 172.21.15.40:0/3002687865 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f5c0af6d0 0x7f0f5c156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.652+0000 7f0f60d53640 1 -- 172.21.15.40:0/3002687865 >> 172.21.15.40:0/3002687865 conn(0x7f0f5c06d700 msgr2=0x7f0f5c15a300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:53.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.652+0000 7f0f60d53640 1 -- 172.21.15.40:0/3002687865 shutdown_connections 2023-10-16T08:38:53.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.652+0000 7f0f60d53640 1 -- 172.21.15.40:0/3002687865 wait complete. 2023-10-16T08:38:53.673 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-10-16T08:38:52.940370+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592778,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219016,"kb_used_data":2824,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755704,"statfs":{"total":773094113280,"available":772869840896,"internally_reserved":0,"allocated":2891776,"data_stored":1752740,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.003099"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931534+0000","last_change":"2023-10-16T08:38:45.650619+0000","last_active":"2023-10-16T08:38:46.931534+0000","last_peered":"2023-10-16T08:38:46.931534+0000","last_clean":"2023-10-16T08:38:46.931534+0000","last_became_active":"2023-10-16T08:38:45.649704+0000","last_became_peered":"2023-10-16T08:38:45.649704+0000","last_unstale":"2023-10-16T08:38:46.931534+0000","last_undegraded":"2023-10-16T08:38:46.931534+0000","last_fullsized":"2023-10-16T08:38:46.931534+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:58:51.599709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626698+0000","last_change":"2023-10-16T08:38:45.657659+0000","last_active":"2023-10-16T08:38:46.626698+0000","last_peered":"2023-10-16T08:38:46.626698+0000","last_clean":"2023-10-16T08:38:46.626698+0000","last_became_active":"2023-10-16T08:38:45.656884+0000","last_became_peered":"2023-10-16T08:38:45.656884+0000","last_unstale":"2023-10-16T08:38:46.626698+0000","last_undegraded":"2023-10-16T08:38:46.626698+0000","last_fullsized":"2023-10-16T08:38:46.626698+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926261+0000","last_change":"2023-10-16T08:38:45.630125+0000","last_active":"2023-10-16T08:38:46.926261+0000","last_peered":"2023-10-16T08:38:46.926261+0000","last_clean":"2023-10-16T08:38:46.926261+0000","last_became_active":"2023-10-16T08:38:45.629297+0000","last_became_peered":"2023-10-16T08:38:45.629297+0000","last_unstale":"2023-10-16T08:38:46.926261+0000","last_undegraded":"2023-10-16T08:38:46.926261+0000","last_fullsized":"2023-10-16T08:38:46.926261+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931028+0000","last_change":"2023-10-16T08:38:45.675996+0000","last_active":"2023-10-16T08:38:46.931028+0000","last_peered":"2023-10-16T08:38:46.931028+0000","last_clean":"2023-10-16T08:38:46.931028+0000","last_became_active":"2023-10-16T08:38:45.675327+0000","last_became_peered":"2023-10-16T08:38:45.675327+0000","last_unstale":"2023-10-16T08:38:46.931028+0000","last_undegraded":"2023-10-16T08:38:46.931028+0000","last_fullsized":"2023-10-16T08:38:46.931028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:33.190218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930419+0000","last_change":"2023-10-16T08:38:45.653550+0000","last_active":"2023-10-16T08:38:46.930419+0000","last_peered":"2023-10-16T08:38:46.930419+0000","last_clean":"2023-10-16T08:38:46.930419+0000","last_became_active":"2023-10-16T08:38:45.652808+0000","last_became_peered":"2023-10-16T08:38:45.652808+0000","last_unstale":"2023-10-16T08:38:46.930419+0000","last_undegraded":"2023-10-16T08:38:46.930419+0000","last_fullsized":"2023-10-16T08:38:46.930419+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930519+0000","last_change":"2023-10-16T08:38:45.675086+0000","last_active":"2023-10-16T08:38:46.930519+0000","last_peered":"2023-10-16T08:38:46.930519+0000","last_clean":"2023-10-16T08:38:46.930519+0000","last_became_active":"2023-10-16T08:38:45.671146+0000","last_became_peered":"2023-10-16T08:38:45.671146+0000","last_unstale":"2023-10-16T08:38:46.930519+0000","last_undegraded":"2023-10-16T08:38:46.930519+0000","last_fullsized":"2023-10-16T08:38:46.930519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931145+0000","last_change":"2023-10-16T08:38:45.652595+0000","last_active":"2023-10-16T08:38:46.931145+0000","last_peered":"2023-10-16T08:38:46.931145+0000","last_clean":"2023-10-16T08:38:46.931145+0000","last_became_active":"2023-10-16T08:38:45.651848+0000","last_became_peered":"2023-10-16T08:38:45.651848+0000","last_unstale":"2023-10-16T08:38:46.931145+0000","last_undegraded":"2023-10-16T08:38:46.931145+0000","last_fullsized":"2023-10-16T08:38:46.931145+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:13.453321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628116+0000","last_change":"2023-10-16T08:38:45.668858+0000","last_active":"2023-10-16T08:38:46.628116+0000","last_peered":"2023-10-16T08:38:46.628116+0000","last_clean":"2023-10-16T08:38:46.628116+0000","last_became_active":"2023-10-16T08:38:45.664702+0000","last_became_peered":"2023-10-16T08:38:45.664702+0000","last_unstale":"2023-10-16T08:38:46.628116+0000","last_undegraded":"2023-10-16T08:38:46.628116+0000","last_fullsized":"2023-10-16T08:38:46.628116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931734+0000","last_change":"2023-10-16T08:38:45.667037+0000","last_active":"2023-10-16T08:38:46.931734+0000","last_peered":"2023-10-16T08:38:46.931734+0000","last_clean":"2023-10-16T08:38:46.931734+0000","last_became_active":"2023-10-16T08:38:45.666474+0000","last_became_peered":"2023-10-16T08:38:45.666474+0000","last_unstale":"2023-10-16T08:38:46.931734+0000","last_undegraded":"2023-10-16T08:38:46.931734+0000","last_fullsized":"2023-10-16T08:38:46.931734+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:11:02.634006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633786+0000","last_change":"2023-10-16T08:38:45.669859+0000","last_active":"2023-10-16T08:38:46.633786+0000","last_peered":"2023-10-16T08:38:46.633786+0000","last_clean":"2023-10-16T08:38:46.633786+0000","last_became_active":"2023-10-16T08:38:45.668278+0000","last_became_peered":"2023-10-16T08:38:45.668278+0000","last_unstale":"2023-10-16T08:38:46.633786+0000","last_undegraded":"2023-10-16T08:38:46.633786+0000","last_fullsized":"2023-10-16T08:38:46.633786+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633464+0000","last_change":"2023-10-16T08:38:45.664567+0000","last_active":"2023-10-16T08:38:46.633464+0000","last_peered":"2023-10-16T08:38:46.633464+0000","last_clean":"2023-10-16T08:38:46.633464+0000","last_became_active":"2023-10-16T08:38:45.663813+0000","last_became_peered":"2023-10-16T08:38:45.663813+0000","last_unstale":"2023-10-16T08:38:46.633464+0000","last_undegraded":"2023-10-16T08:38:46.633464+0000","last_fullsized":"2023-10-16T08:38:46.633464+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.932310+0000","last_change":"2023-10-16T08:38:45.675925+0000","last_active":"2023-10-16T08:38:46.932310+0000","last_peered":"2023-10-16T08:38:46.932310+0000","last_clean":"2023-10-16T08:38:46.932310+0000","last_became_active":"2023-10-16T08:38:45.675320+0000","last_became_peered":"2023-10-16T08:38:45.675320+0000","last_unstale":"2023-10-16T08:38:46.932310+0000","last_undegraded":"2023-10-16T08:38:46.932310+0000","last_fullsized":"2023-10-16T08:38:46.932310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:11.991837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628294+0000","last_change":"2023-10-16T08:38:45.668056+0000","last_active":"2023-10-16T08:38:46.628294+0000","last_peered":"2023-10-16T08:38:46.628294+0000","last_clean":"2023-10-16T08:38:46.628294+0000","last_became_active":"2023-10-16T08:38:45.667428+0000","last_became_peered":"2023-10-16T08:38:45.667428+0000","last_unstale":"2023-10-16T08:38:46.628294+0000","last_undegraded":"2023-10-16T08:38:46.628294+0000","last_fullsized":"2023-10-16T08:38:46.628294+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.925984+0000","last_change":"2023-10-16T08:38:45.637518+0000","last_active":"2023-10-16T08:38:46.925984+0000","last_peered":"2023-10-16T08:38:46.925984+0000","last_clean":"2023-10-16T08:38:46.925984+0000","last_became_active":"2023-10-16T08:38:45.636785+0000","last_became_peered":"2023-10-16T08:38:45.636785+0000","last_unstale":"2023-10-16T08:38:46.925984+0000","last_undegraded":"2023-10-16T08:38:46.925984+0000","last_fullsized":"2023-10-16T08:38:46.925984+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:11.870171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626028+0000","last_change":"2023-10-16T08:38:45.656433+0000","last_active":"2023-10-16T08:38:46.626028+0000","last_peered":"2023-10-16T08:38:46.626028+0000","last_clean":"2023-10-16T08:38:46.626028+0000","last_became_active":"2023-10-16T08:38:45.655395+0000","last_became_peered":"2023-10-16T08:38:45.655395+0000","last_unstale":"2023-10-16T08:38:46.626028+0000","last_undegraded":"2023-10-16T08:38:46.626028+0000","last_fullsized":"2023-10-16T08:38:46.626028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.929381+0000","last_change":"2023-10-16T08:38:45.664192+0000","last_active":"2023-10-16T08:38:46.929381+0000","last_peered":"2023-10-16T08:38:46.929381+0000","last_clean":"2023-10-16T08:38:46.929381+0000","last_became_active":"2023-10-16T08:38:45.663312+0000","last_became_peered":"2023-10-16T08:38:45.663312+0000","last_unstale":"2023-10-16T08:38:46.929381+0000","last_undegraded":"2023-10-16T08:38:46.929381+0000","last_fullsized":"2023-10-16T08:38:46.929381+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930671+0000","last_change":"2023-10-16T08:38:45.679894+0000","last_active":"2023-10-16T08:38:46.930671+0000","last_peered":"2023-10-16T08:38:46.930671+0000","last_clean":"2023-10-16T08:38:46.930671+0000","last_became_active":"2023-10-16T08:38:45.676912+0000","last_became_peered":"2023-10-16T08:38:45.676912+0000","last_unstale":"2023-10-16T08:38:46.930671+0000","last_undegraded":"2023-10-16T08:38:46.930671+0000","last_fullsized":"2023-10-16T08:38:46.930671+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927699+0000","last_change":"2023-10-16T08:38:45.631488+0000","last_active":"2023-10-16T08:38:46.927699+0000","last_peered":"2023-10-16T08:38:46.927699+0000","last_clean":"2023-10-16T08:38:46.927699+0000","last_became_active":"2023-10-16T08:38:45.630641+0000","last_became_peered":"2023-10-16T08:38:45.630641+0000","last_unstale":"2023-10-16T08:38:46.927699+0000","last_undegraded":"2023-10-16T08:38:46.927699+0000","last_fullsized":"2023-10-16T08:38:46.927699+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:11.991837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931188+0000","last_change":"2023-10-16T08:38:45.650087+0000","last_active":"2023-10-16T08:38:46.931188+0000","last_peered":"2023-10-16T08:38:46.931188+0000","last_clean":"2023-10-16T08:38:46.931188+0000","last_became_active":"2023-10-16T08:38:45.649475+0000","last_became_peered":"2023-10-16T08:38:45.649475+0000","last_unstale":"2023-10-16T08:38:46.931188+0000","last_undegraded":"2023-10-16T08:38:46.931188+0000","last_fullsized":"2023-10-16T08:38:46.931188+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:26.292137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633283+0000","last_change":"2023-10-16T08:38:45.667336+0000","last_active":"2023-10-16T08:38:46.633283+0000","last_peered":"2023-10-16T08:38:46.633283+0000","last_clean":"2023-10-16T08:38:46.633283+0000","last_became_active":"2023-10-16T08:38:45.665291+0000","last_became_peered":"2023-10-16T08:38:45.665291+0000","last_unstale":"2023-10-16T08:38:46.633283+0000","last_undegraded":"2023-10-16T08:38:46.633283+0000","last_fullsized":"2023-10-16T08:38:46.633283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926490+0000","last_change":"2023-10-16T08:38:45.654036+0000","last_active":"2023-10-16T08:38:46.926490+0000","last_peered":"2023-10-16T08:38:46.926490+0000","last_clean":"2023-10-16T08:38:46.926490+0000","last_became_active":"2023-10-16T08:38:45.652412+0000","last_became_peered":"2023-10-16T08:38:45.652412+0000","last_unstale":"2023-10-16T08:38:46.926490+0000","last_undegraded":"2023-10-16T08:38:46.926490+0000","last_fullsized":"2023-10-16T08:38:46.926490+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930858+0000","last_change":"2023-10-16T08:38:45.677644+0000","last_active":"2023-10-16T08:38:46.930858+0000","last_peered":"2023-10-16T08:38:46.930858+0000","last_clean":"2023-10-16T08:38:46.930858+0000","last_became_active":"2023-10-16T08:38:45.676844+0000","last_became_peered":"2023-10-16T08:38:45.676844+0000","last_unstale":"2023-10-16T08:38:46.930858+0000","last_undegraded":"2023-10-16T08:38:46.930858+0000","last_fullsized":"2023-10-16T08:38:46.930858+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628178+0000","last_change":"2023-10-16T08:38:45.663771+0000","last_active":"2023-10-16T08:38:46.628178+0000","last_peered":"2023-10-16T08:38:46.628178+0000","last_clean":"2023-10-16T08:38:46.628178+0000","last_became_active":"2023-10-16T08:38:45.662944+0000","last_became_peered":"2023-10-16T08:38:45.662944+0000","last_unstale":"2023-10-16T08:38:46.628178+0000","last_undegraded":"2023-10-16T08:38:46.628178+0000","last_fullsized":"2023-10-16T08:38:46.628178+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.929737+0000","last_change":"2023-10-16T08:38:45.667398+0000","last_active":"2023-10-16T08:38:46.929737+0000","last_peered":"2023-10-16T08:38:46.929737+0000","last_clean":"2023-10-16T08:38:46.929737+0000","last_became_active":"2023-10-16T08:38:45.666641+0000","last_became_peered":"2023-10-16T08:38:45.666641+0000","last_unstale":"2023-10-16T08:38:46.929737+0000","last_undegraded":"2023-10-16T08:38:46.929737+0000","last_fullsized":"2023-10-16T08:38:46.929737+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628798+0000","last_change":"2023-10-16T08:38:45.670526+0000","last_active":"2023-10-16T08:38:46.628798+0000","last_peered":"2023-10-16T08:38:46.628798+0000","last_clean":"2023-10-16T08:38:46.628798+0000","last_became_active":"2023-10-16T08:38:45.669734+0000","last_became_peered":"2023-10-16T08:38:45.669734+0000","last_unstale":"2023-10-16T08:38:46.628798+0000","last_undegraded":"2023-10-16T08:38:46.628798+0000","last_fullsized":"2023-10-16T08:38:46.628798+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633809+0000","last_change":"2023-10-16T08:38:45.670429+0000","last_active":"2023-10-16T08:38:46.633809+0000","last_peered":"2023-10-16T08:38:46.633809+0000","last_clean":"2023-10-16T08:38:46.633809+0000","last_became_active":"2023-10-16T08:38:45.665972+0000","last_became_peered":"2023-10-16T08:38:45.665972+0000","last_unstale":"2023-10-16T08:38:46.633809+0000","last_undegraded":"2023-10-16T08:38:46.633809+0000","last_fullsized":"2023-10-16T08:38:46.633809+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626436+0000","last_change":"2023-10-16T08:38:45.667047+0000","last_active":"2023-10-16T08:38:46.626436+0000","last_peered":"2023-10-16T08:38:46.626436+0000","last_clean":"2023-10-16T08:38:46.626436+0000","last_became_active":"2023-10-16T08:38:45.666403+0000","last_became_peered":"2023-10-16T08:38:45.666403+0000","last_unstale":"2023-10-16T08:38:46.626436+0000","last_undegraded":"2023-10-16T08:38:46.626436+0000","last_fullsized":"2023-10-16T08:38:46.626436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628642+0000","last_change":"2023-10-16T08:38:45.668007+0000","last_active":"2023-10-16T08:38:46.628642+0000","last_peered":"2023-10-16T08:38:46.628642+0000","last_clean":"2023-10-16T08:38:46.628642+0000","last_became_active":"2023-10-16T08:38:45.664572+0000","last_became_peered":"2023-10-16T08:38:45.664572+0000","last_unstale":"2023-10-16T08:38:46.628642+0000","last_undegraded":"2023-10-16T08:38:46.628642+0000","last_fullsized":"2023-10-16T08:38:46.628642+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926222+0000","last_change":"2023-10-16T08:38:45.667378+0000","last_active":"2023-10-16T08:38:46.926222+0000","last_peered":"2023-10-16T08:38:46.926222+0000","last_clean":"2023-10-16T08:38:46.926222+0000","last_became_active":"2023-10-16T08:38:45.666547+0000","last_became_peered":"2023-10-16T08:38:45.666547+0000","last_unstale":"2023-10-16T08:38:46.926222+0000","last_undegraded":"2023-10-16T08:38:46.926222+0000","last_fullsized":"2023-10-16T08:38:46.926222+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931201+0000","last_change":"2023-10-16T08:38:45.669071+0000","last_active":"2023-10-16T08:38:46.931201+0000","last_peered":"2023-10-16T08:38:46.931201+0000","last_clean":"2023-10-16T08:38:46.931201+0000","last_became_active":"2023-10-16T08:38:45.666591+0000","last_became_peered":"2023-10-16T08:38:45.666591+0000","last_unstale":"2023-10-16T08:38:46.931201+0000","last_undegraded":"2023-10-16T08:38:46.931201+0000","last_fullsized":"2023-10-16T08:38:46.931201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:02:39.840175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927293+0000","last_change":"2023-10-16T08:38:45.648288+0000","last_active":"2023-10-16T08:38:46.927293+0000","last_peered":"2023-10-16T08:38:46.927293+0000","last_clean":"2023-10-16T08:38:46.927293+0000","last_became_active":"2023-10-16T08:38:45.647484+0000","last_became_peered":"2023-10-16T08:38:45.647484+0000","last_unstale":"2023-10-16T08:38:46.927293+0000","last_undegraded":"2023-10-16T08:38:46.927293+0000","last_fullsized":"2023-10-16T08:38:46.927293+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:20.779617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.632427+0000","last_change":"2023-10-16T08:38:45.638183+0000","last_active":"2023-10-16T08:38:46.632427+0000","last_peered":"2023-10-16T08:38:46.632427+0000","last_clean":"2023-10-16T08:38:46.632427+0000","last_became_active":"2023-10-16T08:38:45.636762+0000","last_became_peered":"2023-10-16T08:38:45.636762+0000","last_unstale":"2023-10-16T08:38:46.632427+0000","last_undegraded":"2023-10-16T08:38:46.632427+0000","last_fullsized":"2023-10-16T08:38:46.632427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.929995+0000","last_change":"2023-10-16T08:38:45.659464+0000","last_active":"2023-10-16T08:38:46.929995+0000","last_peered":"2023-10-16T08:38:46.929995+0000","last_clean":"2023-10-16T08:38:46.929995+0000","last_became_active":"2023-10-16T08:38:45.658808+0000","last_became_peered":"2023-10-16T08:38:45.658808+0000","last_unstale":"2023-10-16T08:38:46.929995+0000","last_undegraded":"2023-10-16T08:38:46.929995+0000","last_fullsized":"2023-10-16T08:38:46.929995+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:25.386221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628419+0000","last_change":"2023-10-16T08:38:44.644122+0000","last_active":"2023-10-16T08:38:46.628419+0000","last_peered":"2023-10-16T08:38:46.628419+0000","last_clean":"2023-10-16T08:38:46.628419+0000","last_became_active":"2023-10-16T08:38:44.643614+0000","last_became_peered":"2023-10-16T08:38:44.643614+0000","last_unstale":"2023-10-16T08:38:46.628419+0000","last_undegraded":"2023-10-16T08:38:46.628419+0000","last_fullsized":"2023-10-16T08:38:46.628419+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926570+0000","last_change":"2023-10-16T08:38:45.659854+0000","last_active":"2023-10-16T08:38:46.926570+0000","last_peered":"2023-10-16T08:38:46.926570+0000","last_clean":"2023-10-16T08:38:46.926570+0000","last_became_active":"2023-10-16T08:38:45.658409+0000","last_became_peered":"2023-10-16T08:38:45.658409+0000","last_unstale":"2023-10-16T08:38:46.926570+0000","last_undegraded":"2023-10-16T08:38:46.926570+0000","last_fullsized":"2023-10-16T08:38:46.926570+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.632862+0000","last_change":"2023-10-16T08:38:44.658819+0000","last_active":"2023-10-16T08:38:46.632862+0000","last_peered":"2023-10-16T08:38:46.632862+0000","last_clean":"2023-10-16T08:38:46.632862+0000","last_became_active":"2023-10-16T08:38:44.658257+0000","last_became_peered":"2023-10-16T08:38:44.658257+0000","last_unstale":"2023-10-16T08:38:46.632862+0000","last_undegraded":"2023-10-16T08:38:46.632862+0000","last_fullsized":"2023-10-16T08:38:46.632862+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:54.848445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628283+0000","last_change":"2023-10-16T08:38:44.647536+0000","last_active":"2023-10-16T08:38:46.628283+0000","last_peered":"2023-10-16T08:38:46.628283+0000","last_clean":"2023-10-16T08:38:46.628283+0000","last_became_active":"2023-10-16T08:38:44.645931+0000","last_became_peered":"2023-10-16T08:38:44.645931+0000","last_unstale":"2023-10-16T08:38:46.628283+0000","last_undegraded":"2023-10-16T08:38:46.628283+0000","last_fullsized":"2023-10-16T08:38:46.628283+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628223+0000","last_change":"2023-10-16T08:38:45.662855+0000","last_active":"2023-10-16T08:38:46.628223+0000","last_peered":"2023-10-16T08:38:46.628223+0000","last_clean":"2023-10-16T08:38:46.628223+0000","last_became_active":"2023-10-16T08:38:45.662072+0000","last_became_peered":"2023-10-16T08:38:45.662072+0000","last_unstale":"2023-10-16T08:38:46.628223+0000","last_undegraded":"2023-10-16T08:38:46.628223+0000","last_fullsized":"2023-10-16T08:38:46.628223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.929953+0000","last_change":"2023-10-16T08:38:44.657651+0000","last_active":"2023-10-16T08:38:46.929953+0000","last_peered":"2023-10-16T08:38:46.929953+0000","last_clean":"2023-10-16T08:38:46.929953+0000","last_became_active":"2023-10-16T08:38:44.657065+0000","last_became_peered":"2023-10-16T08:38:44.657065+0000","last_unstale":"2023-10-16T08:38:46.929953+0000","last_undegraded":"2023-10-16T08:38:46.929953+0000","last_fullsized":"2023-10-16T08:38:46.929953+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.625975+0000","last_change":"2023-10-16T08:38:45.668048+0000","last_active":"2023-10-16T08:38:46.625975+0000","last_peered":"2023-10-16T08:38:46.625975+0000","last_clean":"2023-10-16T08:38:46.625975+0000","last_became_active":"2023-10-16T08:38:45.667413+0000","last_became_peered":"2023-10-16T08:38:45.667413+0000","last_unstale":"2023-10-16T08:38:46.625975+0000","last_undegraded":"2023-10-16T08:38:46.625975+0000","last_fullsized":"2023-10-16T08:38:46.625975+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633561+0000","last_change":"2023-10-16T08:38:44.663188+0000","last_active":"2023-10-16T08:38:46.633561+0000","last_peered":"2023-10-16T08:38:46.633561+0000","last_clean":"2023-10-16T08:38:46.633561+0000","last_became_active":"2023-10-16T08:38:44.662383+0000","last_became_peered":"2023-10-16T08:38:44.662383+0000","last_unstale":"2023-10-16T08:38:46.633561+0000","last_undegraded":"2023-10-16T08:38:46.633561+0000","last_fullsized":"2023-10-16T08:38:46.633561+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:24.375366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930913+0000","last_change":"2023-10-16T08:38:45.679271+0000","last_active":"2023-10-16T08:38:46.930913+0000","last_peered":"2023-10-16T08:38:46.930913+0000","last_clean":"2023-10-16T08:38:46.930913+0000","last_became_active":"2023-10-16T08:38:45.678050+0000","last_became_peered":"2023-10-16T08:38:45.678050+0000","last_unstale":"2023-10-16T08:38:46.930913+0000","last_undegraded":"2023-10-16T08:38:46.930913+0000","last_fullsized":"2023-10-16T08:38:46.930913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931150+0000","last_change":"2023-10-16T08:38:44.653478+0000","last_active":"2023-10-16T08:38:46.931150+0000","last_peered":"2023-10-16T08:38:46.931150+0000","last_clean":"2023-10-16T08:38:46.931150+0000","last_became_active":"2023-10-16T08:38:44.652618+0000","last_became_peered":"2023-10-16T08:38:44.652618+0000","last_unstale":"2023-10-16T08:38:46.931150+0000","last_undegraded":"2023-10-16T08:38:46.931150+0000","last_fullsized":"2023-10-16T08:38:46.931150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926242+0000","last_change":"2023-10-16T08:38:45.656603+0000","last_active":"2023-10-16T08:38:46.926242+0000","last_peered":"2023-10-16T08:38:46.926242+0000","last_clean":"2023-10-16T08:38:46.926242+0000","last_became_active":"2023-10-16T08:38:45.655213+0000","last_became_peered":"2023-10-16T08:38:45.655213+0000","last_unstale":"2023-10-16T08:38:46.926242+0000","last_undegraded":"2023-10-16T08:38:46.926242+0000","last_fullsized":"2023-10-16T08:38:46.926242+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626622+0000","last_change":"2023-10-16T08:38:44.651637+0000","last_active":"2023-10-16T08:38:46.626622+0000","last_peered":"2023-10-16T08:38:46.626622+0000","last_clean":"2023-10-16T08:38:46.626622+0000","last_became_active":"2023-10-16T08:38:44.651359+0000","last_became_peered":"2023-10-16T08:38:44.651359+0000","last_unstale":"2023-10-16T08:38:46.626622+0000","last_undegraded":"2023-10-16T08:38:46.626622+0000","last_fullsized":"2023-10-16T08:38:46.626622+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:10.859316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.932633+0000","last_change":"2023-10-16T08:38:45.661741+0000","last_active":"2023-10-16T08:38:46.932633+0000","last_peered":"2023-10-16T08:38:46.932633+0000","last_clean":"2023-10-16T08:38:46.932633+0000","last_became_active":"2023-10-16T08:38:45.660611+0000","last_became_peered":"2023-10-16T08:38:45.660611+0000","last_unstale":"2023-10-16T08:38:46.932633+0000","last_undegraded":"2023-10-16T08:38:46.932633+0000","last_fullsized":"2023-10-16T08:38:46.932633+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:55.859300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"19'6","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633359+0000","last_change":"2023-10-16T08:38:44.647593+0000","last_active":"2023-10-16T08:38:46.633359+0000","last_peered":"2023-10-16T08:38:46.633359+0000","last_clean":"2023-10-16T08:38:46.633359+0000","last_became_active":"2023-10-16T08:38:44.647001+0000","last_became_peered":"2023-10-16T08:38:44.647001+0000","last_unstale":"2023-10-16T08:38:46.633359+0000","last_undegraded":"2023-10-16T08:38:46.633359+0000","last_fullsized":"2023-10-16T08:38:46.633359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1612,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930291+0000","last_change":"2023-10-16T08:38:45.668629+0000","last_active":"2023-10-16T08:38:46.930291+0000","last_peered":"2023-10-16T08:38:46.930291+0000","last_clean":"2023-10-16T08:38:46.930291+0000","last_became_active":"2023-10-16T08:38:45.667334+0000","last_became_peered":"2023-10-16T08:38:45.667334+0000","last_unstale":"2023-10-16T08:38:46.930291+0000","last_undegraded":"2023-10-16T08:38:46.930291+0000","last_fullsized":"2023-10-16T08:38:46.930291+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926736+0000","last_change":"2023-10-16T08:38:44.655832+0000","last_active":"2023-10-16T08:38:46.926736+0000","last_peered":"2023-10-16T08:38:46.926736+0000","last_clean":"2023-10-16T08:38:46.926736+0000","last_became_active":"2023-10-16T08:38:44.655469+0000","last_became_peered":"2023-10-16T08:38:44.655469+0000","last_unstale":"2023-10-16T08:38:46.926736+0000","last_undegraded":"2023-10-16T08:38:46.926736+0000","last_fullsized":"2023-10-16T08:38:46.926736+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:19.768762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926112+0000","last_change":"2023-10-16T08:38:45.653688+0000","last_active":"2023-10-16T08:38:46.926112+0000","last_peered":"2023-10-16T08:38:46.926112+0000","last_clean":"2023-10-16T08:38:46.926112+0000","last_became_active":"2023-10-16T08:38:45.650563+0000","last_became_peered":"2023-10-16T08:38:45.650563+0000","last_unstale":"2023-10-16T08:38:46.926112+0000","last_undegraded":"2023-10-16T08:38:46.926112+0000","last_fullsized":"2023-10-16T08:38:46.926112+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:29.539285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931330+0000","last_change":"2023-10-16T08:38:44.662816+0000","last_active":"2023-10-16T08:38:46.931330+0000","last_peered":"2023-10-16T08:38:46.931330+0000","last_clean":"2023-10-16T08:38:46.931330+0000","last_became_active":"2023-10-16T08:38:44.662110+0000","last_became_peered":"2023-10-16T08:38:44.662110+0000","last_unstale":"2023-10-16T08:38:46.931330+0000","last_undegraded":"2023-10-16T08:38:46.931330+0000","last_fullsized":"2023-10-16T08:38:46.931330+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:00.201724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930788+0000","last_change":"2023-10-16T08:38:45.662166+0000","last_active":"2023-10-16T08:38:46.930788+0000","last_peered":"2023-10-16T08:38:46.930788+0000","last_clean":"2023-10-16T08:38:46.930788+0000","last_became_active":"2023-10-16T08:38:45.661595+0000","last_became_peered":"2023-10-16T08:38:45.661595+0000","last_unstale":"2023-10-16T08:38:46.930788+0000","last_undegraded":"2023-10-16T08:38:46.930788+0000","last_fullsized":"2023-10-16T08:38:46.930788+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:02:39.840175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.632731+0000","last_change":"2023-10-16T08:38:44.657766+0000","last_active":"2023-10-16T08:38:46.632731+0000","last_peered":"2023-10-16T08:38:46.632731+0000","last_clean":"2023-10-16T08:38:46.632731+0000","last_became_active":"2023-10-16T08:38:44.657153+0000","last_became_peered":"2023-10-16T08:38:44.657153+0000","last_unstale":"2023-10-16T08:38:46.632731+0000","last_undegraded":"2023-10-16T08:38:46.632731+0000","last_fullsized":"2023-10-16T08:38:46.632731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.632435+0000","last_change":"2023-10-16T08:38:45.637591+0000","last_active":"2023-10-16T08:38:46.632435+0000","last_peered":"2023-10-16T08:38:46.632435+0000","last_clean":"2023-10-16T08:38:46.632435+0000","last_became_active":"2023-10-16T08:38:45.635633+0000","last_became_peered":"2023-10-16T08:38:45.635633+0000","last_unstale":"2023-10-16T08:38:46.632435+0000","last_undegraded":"2023-10-16T08:38:46.632435+0000","last_fullsized":"2023-10-16T08:38:46.632435+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":20,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927363+0000","last_change":"2023-10-16T08:38:44.638297+0000","last_active":"2023-10-16T08:38:46.927363+0000","last_peered":"2023-10-16T08:38:46.927363+0000","last_clean":"2023-10-16T08:38:46.927363+0000","last_became_active":"2023-10-16T08:38:44.637640+0000","last_became_peered":"2023-10-16T08:38:44.637640+0000","last_unstale":"2023-10-16T08:38:46.927363+0000","last_undegraded":"2023-10-16T08:38:46.927363+0000","last_fullsized":"2023-10-16T08:38:46.927363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:16.003943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931498+0000","last_change":"2023-10-16T08:38:45.657286+0000","last_active":"2023-10-16T08:38:46.931498+0000","last_peered":"2023-10-16T08:38:46.931498+0000","last_clean":"2023-10-16T08:38:46.931498+0000","last_became_active":"2023-10-16T08:38:45.656651+0000","last_became_peered":"2023-10-16T08:38:45.656651+0000","last_unstale":"2023-10-16T08:38:46.931498+0000","last_undegraded":"2023-10-16T08:38:46.931498+0000","last_fullsized":"2023-10-16T08:38:46.931498+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:28.052483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.932737+0000","last_change":"2023-10-16T08:38:44.648434+0000","last_active":"2023-10-16T08:38:46.932737+0000","last_peered":"2023-10-16T08:38:46.932737+0000","last_clean":"2023-10-16T08:38:46.932737+0000","last_became_active":"2023-10-16T08:38:44.647887+0000","last_became_peered":"2023-10-16T08:38:44.647887+0000","last_unstale":"2023-10-16T08:38:46.932737+0000","last_undegraded":"2023-10-16T08:38:46.932737+0000","last_fullsized":"2023-10-16T08:38:46.932737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930414+0000","last_change":"2023-10-16T08:38:45.661965+0000","last_active":"2023-10-16T08:38:46.930414+0000","last_peered":"2023-10-16T08:38:46.930414+0000","last_clean":"2023-10-16T08:38:46.930414+0000","last_became_active":"2023-10-16T08:38:45.656750+0000","last_became_peered":"2023-10-16T08:38:45.656750+0000","last_unstale":"2023-10-16T08:38:46.930414+0000","last_undegraded":"2023-10-16T08:38:46.930414+0000","last_fullsized":"2023-10-16T08:38:46.930414+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927746+0000","last_change":"2023-10-16T08:38:44.647524+0000","last_active":"2023-10-16T08:38:46.927746+0000","last_peered":"2023-10-16T08:38:46.927746+0000","last_clean":"2023-10-16T08:38:46.927746+0000","last_became_active":"2023-10-16T08:38:44.646042+0000","last_became_peered":"2023-10-16T08:38:44.646042+0000","last_unstale":"2023-10-16T08:38:46.927746+0000","last_undegraded":"2023-10-16T08:38:46.927746+0000","last_fullsized":"2023-10-16T08:38:46.927746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:24.375366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931909+0000","last_change":"2023-10-16T08:38:45.670969+0000","last_active":"2023-10-16T08:38:46.931909+0000","last_peered":"2023-10-16T08:38:46.931909+0000","last_clean":"2023-10-16T08:38:46.931909+0000","last_became_active":"2023-10-16T08:38:45.670415+0000","last_became_peered":"2023-10-16T08:38:45.670415+0000","last_unstale":"2023-10-16T08:38:46.931909+0000","last_undegraded":"2023-10-16T08:38:46.931909+0000","last_fullsized":"2023-10-16T08:38:46.931909+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:15:00.472153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626608+0000","last_change":"2023-10-16T08:38:44.655744+0000","last_active":"2023-10-16T08:38:46.626608+0000","last_peered":"2023-10-16T08:38:46.626608+0000","last_clean":"2023-10-16T08:38:46.626608+0000","last_became_active":"2023-10-16T08:38:44.655260+0000","last_became_peered":"2023-10-16T08:38:44.655260+0000","last_unstale":"2023-10-16T08:38:46.626608+0000","last_undegraded":"2023-10-16T08:38:46.626608+0000","last_fullsized":"2023-10-16T08:38:46.626608+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:54.848445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931710+0000","last_change":"2023-10-16T08:38:45.668230+0000","last_active":"2023-10-16T08:38:46.931710+0000","last_peered":"2023-10-16T08:38:46.931710+0000","last_clean":"2023-10-16T08:38:46.931710+0000","last_became_active":"2023-10-16T08:38:45.667627+0000","last_became_peered":"2023-10-16T08:38:45.667627+0000","last_unstale":"2023-10-16T08:38:46.931710+0000","last_undegraded":"2023-10-16T08:38:46.931710+0000","last_fullsized":"2023-10-16T08:38:46.931710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:28.052483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927508+0000","last_change":"2023-10-16T08:38:44.662118+0000","last_active":"2023-10-16T08:38:46.927508+0000","last_peered":"2023-10-16T08:38:46.927508+0000","last_clean":"2023-10-16T08:38:46.927508+0000","last_became_active":"2023-10-16T08:38:44.661736+0000","last_became_peered":"2023-10-16T08:38:44.661736+0000","last_unstale":"2023-10-16T08:38:46.927508+0000","last_undegraded":"2023-10-16T08:38:46.927508+0000","last_fullsized":"2023-10-16T08:38:46.927508+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:49:40.967349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930619+0000","last_change":"2023-10-16T08:38:45.667423+0000","last_active":"2023-10-16T08:38:46.930619+0000","last_peered":"2023-10-16T08:38:46.930619+0000","last_clean":"2023-10-16T08:38:46.930619+0000","last_became_active":"2023-10-16T08:38:45.666390+0000","last_became_peered":"2023-10-16T08:38:45.666390+0000","last_unstale":"2023-10-16T08:38:46.930619+0000","last_undegraded":"2023-10-16T08:38:46.930619+0000","last_fullsized":"2023-10-16T08:38:46.930619+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:33.190218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":165,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-10-16T08:38:45.628438+0000","last_change":"2023-10-16T08:38:34.313511+0000","last_active":"2023-10-16T08:38:45.628438+0000","last_peered":"2023-10-16T08:38:45.628438+0000","last_clean":"2023-10-16T08:38:45.628438+0000","last_became_active":"2023-10-16T08:38:34.310944+0000","last_became_peered":"2023-10-16T08:38:34.310944+0000","last_unstale":"2023-10-16T08:38:45.628438+0000","last_undegraded":"2023-10-16T08:38:45.628438+0000","last_fullsized":"2023-10-16T08:38:45.628438+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:33.264692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:33.264692+0000","last_clean_scrub_stamp":"2023-10-16T08:38:33.264692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:50.170461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930271+0000","last_change":"2023-10-16T08:38:44.655223+0000","last_active":"2023-10-16T08:38:46.930271+0000","last_peered":"2023-10-16T08:38:46.930271+0000","last_clean":"2023-10-16T08:38:46.930271+0000","last_became_active":"2023-10-16T08:38:44.654277+0000","last_became_peered":"2023-10-16T08:38:44.654277+0000","last_unstale":"2023-10-16T08:38:46.930271+0000","last_undegraded":"2023-10-16T08:38:46.930271+0000","last_fullsized":"2023-10-16T08:38:46.930271+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:03.462433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628392+0000","last_change":"2023-10-16T08:38:45.679868+0000","last_active":"2023-10-16T08:38:46.628392+0000","last_peered":"2023-10-16T08:38:46.628392+0000","last_clean":"2023-10-16T08:38:46.628392+0000","last_became_active":"2023-10-16T08:38:45.679409+0000","last_became_peered":"2023-10-16T08:38:45.679409+0000","last_unstale":"2023-10-16T08:38:46.628392+0000","last_undegraded":"2023-10-16T08:38:46.628392+0000","last_fullsized":"2023-10-16T08:38:46.628392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930490+0000","last_change":"2023-10-16T08:38:44.652152+0000","last_active":"2023-10-16T08:38:46.930490+0000","last_peered":"2023-10-16T08:38:46.930490+0000","last_clean":"2023-10-16T08:38:46.930490+0000","last_became_active":"2023-10-16T08:38:44.651534+0000","last_became_peered":"2023-10-16T08:38:44.651534+0000","last_unstale":"2023-10-16T08:38:46.930490+0000","last_undegraded":"2023-10-16T08:38:46.930490+0000","last_fullsized":"2023-10-16T08:38:46.930490+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926115+0000","last_change":"2023-10-16T08:38:45.651786+0000","last_active":"2023-10-16T08:38:46.926115+0000","last_peered":"2023-10-16T08:38:46.926115+0000","last_clean":"2023-10-16T08:38:46.926115+0000","last_became_active":"2023-10-16T08:38:45.650715+0000","last_became_peered":"2023-10-16T08:38:45.650715+0000","last_unstale":"2023-10-16T08:38:46.926115+0000","last_undegraded":"2023-10-16T08:38:46.926115+0000","last_fullsized":"2023-10-16T08:38:46.926115+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:29.799799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.927326+0000","last_change":"2023-10-16T08:38:44.645933+0000","last_active":"2023-10-16T08:38:46.927326+0000","last_peered":"2023-10-16T08:38:46.927326+0000","last_clean":"2023-10-16T08:38:46.927326+0000","last_became_active":"2023-10-16T08:38:44.645451+0000","last_became_peered":"2023-10-16T08:38:44.645451+0000","last_unstale":"2023-10-16T08:38:46.927326+0000","last_undegraded":"2023-10-16T08:38:46.927326+0000","last_fullsized":"2023-10-16T08:38:46.927326+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.625977+0000","last_change":"2023-10-16T08:38:45.641499+0000","last_active":"2023-10-16T08:38:46.625977+0000","last_peered":"2023-10-16T08:38:46.625977+0000","last_clean":"2023-10-16T08:38:46.625977+0000","last_became_active":"2023-10-16T08:38:45.640765+0000","last_became_peered":"2023-10-16T08:38:45.640765+0000","last_unstale":"2023-10-16T08:38:46.625977+0000","last_undegraded":"2023-10-16T08:38:46.625977+0000","last_fullsized":"2023-10-16T08:38:46.625977+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931417+0000","last_change":"2023-10-16T08:38:44.655401+0000","last_active":"2023-10-16T08:38:46.931417+0000","last_peered":"2023-10-16T08:38:46.931417+0000","last_clean":"2023-10-16T08:38:46.931417+0000","last_became_active":"2023-10-16T08:38:44.654622+0000","last_became_peered":"2023-10-16T08:38:44.654622+0000","last_unstale":"2023-10-16T08:38:46.931417+0000","last_undegraded":"2023-10-16T08:38:46.931417+0000","last_fullsized":"2023-10-16T08:38:46.931417+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:48.705915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.932454+0000","last_change":"2023-10-16T08:38:45.659377+0000","last_active":"2023-10-16T08:38:46.932454+0000","last_peered":"2023-10-16T08:38:46.932454+0000","last_clean":"2023-10-16T08:38:46.932454+0000","last_became_active":"2023-10-16T08:38:45.658594+0000","last_became_peered":"2023-10-16T08:38:45.658594+0000","last_unstale":"2023-10-16T08:38:46.932454+0000","last_undegraded":"2023-10-16T08:38:46.932454+0000","last_fullsized":"2023-10-16T08:38:46.932454+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:56.738058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626155+0000","last_change":"2023-10-16T08:38:44.634158+0000","last_active":"2023-10-16T08:38:46.626155+0000","last_peered":"2023-10-16T08:38:46.626155+0000","last_clean":"2023-10-16T08:38:46.626155+0000","last_became_active":"2023-10-16T08:38:44.633854+0000","last_became_peered":"2023-10-16T08:38:44.633854+0000","last_unstale":"2023-10-16T08:38:46.626155+0000","last_undegraded":"2023-10-16T08:38:46.626155+0000","last_fullsized":"2023-10-16T08:38:46.626155+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:55.727203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931328+0000","last_change":"2023-10-16T08:38:45.675931+0000","last_active":"2023-10-16T08:38:46.931328+0000","last_peered":"2023-10-16T08:38:46.931328+0000","last_clean":"2023-10-16T08:38:46.931328+0000","last_became_active":"2023-10-16T08:38:45.674962+0000","last_became_peered":"2023-10-16T08:38:45.674962+0000","last_unstale":"2023-10-16T08:38:46.931328+0000","last_undegraded":"2023-10-16T08:38:46.931328+0000","last_fullsized":"2023-10-16T08:38:46.931328+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:13.453321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931397+0000","last_change":"2023-10-16T08:38:44.655693+0000","last_active":"2023-10-16T08:38:46.931397+0000","last_peered":"2023-10-16T08:38:46.931397+0000","last_clean":"2023-10-16T08:38:46.931397+0000","last_became_active":"2023-10-16T08:38:44.655001+0000","last_became_peered":"2023-10-16T08:38:44.655001+0000","last_unstale":"2023-10-16T08:38:46.931397+0000","last_undegraded":"2023-10-16T08:38:46.931397+0000","last_fullsized":"2023-10-16T08:38:46.931397+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:30.649639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626084+0000","last_change":"2023-10-16T08:38:45.639658+0000","last_active":"2023-10-16T08:38:46.626084+0000","last_peered":"2023-10-16T08:38:46.626084+0000","last_clean":"2023-10-16T08:38:46.626084+0000","last_became_active":"2023-10-16T08:38:45.638702+0000","last_became_peered":"2023-10-16T08:38:45.638702+0000","last_unstale":"2023-10-16T08:38:46.626084+0000","last_undegraded":"2023-10-16T08:38:46.626084+0000","last_fullsized":"2023-10-16T08:38:46.626084+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:47.979511+0000","last_change":"2023-10-16T08:38:44.648715+0000","last_active":"2023-10-16T08:38:47.979511+0000","last_peered":"2023-10-16T08:38:47.979511+0000","last_clean":"2023-10-16T08:38:47.979511+0000","last_became_active":"2023-10-16T08:38:44.648312+0000","last_became_peered":"2023-10-16T08:38:44.648312+0000","last_unstale":"2023-10-16T08:38:47.979511+0000","last_undegraded":"2023-10-16T08:38:47.979511+0000","last_fullsized":"2023-10-16T08:38:47.979511+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628807+0000","last_change":"2023-10-16T08:38:45.663402+0000","last_active":"2023-10-16T08:38:46.628807+0000","last_peered":"2023-10-16T08:38:46.628807+0000","last_clean":"2023-10-16T08:38:46.628807+0000","last_became_active":"2023-10-16T08:38:45.662442+0000","last_became_peered":"2023-10-16T08:38:45.662442+0000","last_unstale":"2023-10-16T08:38:46.628807+0000","last_undegraded":"2023-10-16T08:38:46.628807+0000","last_fullsized":"2023-10-16T08:38:46.628807+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.633639+0000","last_change":"2023-10-16T08:38:44.650015+0000","last_active":"2023-10-16T08:38:46.633639+0000","last_peered":"2023-10-16T08:38:46.633639+0000","last_clean":"2023-10-16T08:38:46.633639+0000","last_became_active":"2023-10-16T08:38:44.648155+0000","last_became_peered":"2023-10-16T08:38:44.648155+0000","last_unstale":"2023-10-16T08:38:46.633639+0000","last_undegraded":"2023-10-16T08:38:46.633639+0000","last_fullsized":"2023-10-16T08:38:46.633639+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926726+0000","last_change":"2023-10-16T08:38:45.637568+0000","last_active":"2023-10-16T08:38:46.926726+0000","last_peered":"2023-10-16T08:38:46.926726+0000","last_clean":"2023-10-16T08:38:46.926726+0000","last_became_active":"2023-10-16T08:38:45.636751+0000","last_became_peered":"2023-10-16T08:38:45.636751+0000","last_unstale":"2023-10-16T08:38:46.926726+0000","last_undegraded":"2023-10-16T08:38:46.926726+0000","last_fullsized":"2023-10-16T08:38:46.926726+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930247+0000","last_change":"2023-10-16T08:38:44.649257+0000","last_active":"2023-10-16T08:38:46.930247+0000","last_peered":"2023-10-16T08:38:46.930247+0000","last_clean":"2023-10-16T08:38:46.930247+0000","last_became_active":"2023-10-16T08:38:44.646990+0000","last_became_peered":"2023-10-16T08:38:44.646990+0000","last_unstale":"2023-10-16T08:38:46.930247+0000","last_undegraded":"2023-10-16T08:38:46.930247+0000","last_fullsized":"2023-10-16T08:38:46.930247+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930389+0000","last_change":"2023-10-16T08:38:45.648228+0000","last_active":"2023-10-16T08:38:46.930389+0000","last_peered":"2023-10-16T08:38:46.930389+0000","last_clean":"2023-10-16T08:38:46.930389+0000","last_became_active":"2023-10-16T08:38:45.647449+0000","last_became_peered":"2023-10-16T08:38:45.647449+0000","last_unstale":"2023-10-16T08:38:46.930389+0000","last_undegraded":"2023-10-16T08:38:46.930389+0000","last_fullsized":"2023-10-16T08:38:46.930389+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:20.779617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626045+0000","last_change":"2023-10-16T08:38:44.631354+0000","last_active":"2023-10-16T08:38:46.626045+0000","last_peered":"2023-10-16T08:38:46.626045+0000","last_clean":"2023-10-16T08:38:46.626045+0000","last_became_active":"2023-10-16T08:38:44.630451+0000","last_became_peered":"2023-10-16T08:38:44.630451+0000","last_unstale":"2023-10-16T08:38:46.626045+0000","last_undegraded":"2023-10-16T08:38:46.626045+0000","last_fullsized":"2023-10-16T08:38:46.626045+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:10.980982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628323+0000","last_change":"2023-10-16T08:38:45.663898+0000","last_active":"2023-10-16T08:38:46.628323+0000","last_peered":"2023-10-16T08:38:46.628323+0000","last_clean":"2023-10-16T08:38:46.628323+0000","last_became_active":"2023-10-16T08:38:45.663100+0000","last_became_peered":"2023-10-16T08:38:45.663100+0000","last_unstale":"2023-10-16T08:38:46.628323+0000","last_undegraded":"2023-10-16T08:38:46.628323+0000","last_fullsized":"2023-10-16T08:38:46.628323+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.626257+0000","last_change":"2023-10-16T08:38:44.646987+0000","last_active":"2023-10-16T08:38:46.626257+0000","last_peered":"2023-10-16T08:38:46.626257+0000","last_clean":"2023-10-16T08:38:46.626257+0000","last_became_active":"2023-10-16T08:38:44.646580+0000","last_became_peered":"2023-10-16T08:38:44.646580+0000","last_unstale":"2023-10-16T08:38:46.626257+0000","last_undegraded":"2023-10-16T08:38:46.626257+0000","last_fullsized":"2023-10-16T08:38:46.626257+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628448+0000","last_change":"2023-10-16T08:38:45.662894+0000","last_active":"2023-10-16T08:38:46.628448+0000","last_peered":"2023-10-16T08:38:46.628448+0000","last_clean":"2023-10-16T08:38:46.628448+0000","last_became_active":"2023-10-16T08:38:45.662226+0000","last_became_peered":"2023-10-16T08:38:45.662226+0000","last_unstale":"2023-10-16T08:38:46.628448+0000","last_undegraded":"2023-10-16T08:38:46.628448+0000","last_fullsized":"2023-10-16T08:38:46.628448+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931043+0000","last_change":"2023-10-16T08:38:44.660571+0000","last_active":"2023-10-16T08:38:46.931043+0000","last_peered":"2023-10-16T08:38:46.931043+0000","last_clean":"2023-10-16T08:38:46.931043+0000","last_became_active":"2023-10-16T08:38:44.659976+0000","last_became_peered":"2023-10-16T08:38:44.659976+0000","last_unstale":"2023-10-16T08:38:46.931043+0000","last_undegraded":"2023-10-16T08:38:46.931043+0000","last_fullsized":"2023-10-16T08:38:46.931043+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.632966+0000","last_change":"2023-10-16T08:38:45.665200+0000","last_active":"2023-10-16T08:38:46.632966+0000","last_peered":"2023-10-16T08:38:46.632966+0000","last_clean":"2023-10-16T08:38:46.632966+0000","last_became_active":"2023-10-16T08:38:45.664616+0000","last_became_peered":"2023-10-16T08:38:45.664616+0000","last_unstale":"2023-10-16T08:38:46.632966+0000","last_undegraded":"2023-10-16T08:38:46.632966+0000","last_fullsized":"2023-10-16T08:38:46.632966+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.933149+0000","last_change":"2023-10-16T08:38:45.680892+0000","last_active":"2023-10-16T08:38:46.933149+0000","last_peered":"2023-10-16T08:38:46.933149+0000","last_clean":"2023-10-16T08:38:46.933149+0000","last_became_active":"2023-10-16T08:38:45.680231+0000","last_became_peered":"2023-10-16T08:38:45.680231+0000","last_unstale":"2023-10-16T08:38:46.933149+0000","last_undegraded":"2023-10-16T08:38:46.933149+0000","last_fullsized":"2023-10-16T08:38:46.933149+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:58:51.599709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.929904+0000","last_change":"2023-10-16T08:38:44.664269+0000","last_active":"2023-10-16T08:38:46.929904+0000","last_peered":"2023-10-16T08:38:46.929904+0000","last_clean":"2023-10-16T08:38:46.929904+0000","last_became_active":"2023-10-16T08:38:44.663674+0000","last_became_peered":"2023-10-16T08:38:44.663674+0000","last_unstale":"2023-10-16T08:38:46.929904+0000","last_undegraded":"2023-10-16T08:38:46.929904+0000","last_fullsized":"2023-10-16T08:38:46.929904+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.931372+0000","last_change":"2023-10-16T08:38:45.680890+0000","last_active":"2023-10-16T08:38:46.931372+0000","last_peered":"2023-10-16T08:38:46.931372+0000","last_clean":"2023-10-16T08:38:46.931372+0000","last_became_active":"2023-10-16T08:38:45.680266+0000","last_became_peered":"2023-10-16T08:38:45.680266+0000","last_unstale":"2023-10-16T08:38:46.931372+0000","last_undegraded":"2023-10-16T08:38:46.931372+0000","last_fullsized":"2023-10-16T08:38:46.931372+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:26.292137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.930850+0000","last_change":"2023-10-16T08:38:44.653447+0000","last_active":"2023-10-16T08:38:46.930850+0000","last_peered":"2023-10-16T08:38:46.930850+0000","last_clean":"2023-10-16T08:38:46.930850+0000","last_became_active":"2023-10-16T08:38:44.652658+0000","last_became_peered":"2023-10-16T08:38:44.652658+0000","last_unstale":"2023-10-16T08:38:46.930850+0000","last_undegraded":"2023-10-16T08:38:46.930850+0000","last_fullsized":"2023-10-16T08:38:46.930850+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:22.350066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926151+0000","last_change":"2023-10-16T08:38:45.665041+0000","last_active":"2023-10-16T08:38:46.926151+0000","last_peered":"2023-10-16T08:38:46.926151+0000","last_clean":"2023-10-16T08:38:46.926151+0000","last_became_active":"2023-10-16T08:38:45.664274+0000","last_became_peered":"2023-10-16T08:38:45.664274+0000","last_unstale":"2023-10-16T08:38:46.926151+0000","last_undegraded":"2023-10-16T08:38:46.926151+0000","last_fullsized":"2023-10-16T08:38:46.926151+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628731+0000","last_change":"2023-10-16T08:38:44.645754+0000","last_active":"2023-10-16T08:38:46.628731+0000","last_peered":"2023-10-16T08:38:46.628731+0000","last_clean":"2023-10-16T08:38:46.628731+0000","last_became_active":"2023-10-16T08:38:44.645000+0000","last_became_peered":"2023-10-16T08:38:44.645000+0000","last_unstale":"2023-10-16T08:38:46.628731+0000","last_undegraded":"2023-10-16T08:38:46.628731+0000","last_fullsized":"2023-10-16T08:38:46.628731+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.628628+0000","last_change":"2023-10-16T08:38:45.664252+0000","last_active":"2023-10-16T08:38:46.628628+0000","last_peered":"2023-10-16T08:38:46.628628+0000","last_clean":"2023-10-16T08:38:46.628628+0000","last_became_active":"2023-10-16T08:38:45.663538+0000","last_became_peered":"2023-10-16T08:38:45.663538+0000","last_unstale":"2023-10-16T08:38:46.628628+0000","last_undegraded":"2023-10-16T08:38:46.628628+0000","last_fullsized":"2023-10-16T08:38:46.628628+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-10-16T08:38:46.926310+0000","last_change":"2023-10-16T08:38:44.646321+0000","last_active":"2023-10-16T08:38:46.926310+0000","last_peered":"2023-10-16T08:38:46.926310+0000","last_clean":"2023-10-16T08:38:46.926310+0000","last_became_active":"2023-10-16T08:38:44.645854+0000","last_became_peered":"2023-10-16T08:38:44.645854+0000","last_unstale":"2023-10-16T08:38:46.926310+0000","last_undegraded":"2023-10-16T08:38:46.926310+0000","last_fullsized":"2023-10-16T08:38:46.926310+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:10.980982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2410,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4820,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27808,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344032,"statfs":{"total":96636764160,"available":96608288768,"internally_reserved":0,"allocated":802816,"data_stored":661848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81299999999999994}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.875}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68600000000000005}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91200000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.125}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98499999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95199999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.99299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.90100000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0760000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.429}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27228,"kb_used_data":204,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344612,"statfs":{"total":96636764160,"available":96608882688,"internally_reserved":0,"allocated":208896,"data_stored":70988,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.622}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85199999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97999999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0840000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":71024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.276}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.056}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.2390000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1970000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.159}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27236,"kb_used_data":212,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344604,"statfs":{"total":96636764160,"available":96608874496,"internally_reserved":0,"allocated":217088,"data_stored":72510,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85599999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.149}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.393}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.454}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.534}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27224,"kb_used_data":200,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344616,"statfs":{"total":96636764160,"available":96608886784,"internally_reserved":0,"allocated":204800,"data_stored":70898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41799999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.31900000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70599999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81699999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27232,"kb_used_data":208,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344608,"statfs":{"total":96636764160,"available":96608878592,"internally_reserved":0,"allocated":212992,"data_stored":71058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46300000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84099999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344024,"statfs":{"total":96636764160,"available":96608280576,"internally_reserved":0,"allocated":811008,"data_stored":661904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57499999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59199999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:38:53.678 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-16T08:38:53.678 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-16T08:38:53.679 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-16T08:38:53.679 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-16T08:38:53.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.816+0000 7f3c5d965640 1 Processor -- start 2023-10-16T08:38:53.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.816+0000 7f3c5d965640 1 -- start start 2023-10-16T08:38:53.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.816+0000 7f3c5d965640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3c5815b040 0x7f3c580ca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:53.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.816+0000 7f3c5d965640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c580adec0 0x7f3c580cac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:53.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.816+0000 7f3c5d965640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c580adca0 con 0x7f3c580c8fb0 2023-10-16T08:38:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.816+0000 7f3c5d965640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c580c5840 con 0x7f3c580adec0 2023-10-16T08:38:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.816+0000 7f3c5d965640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c580cf890 con 0x7f3c5815b040 2023-10-16T08:38:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c567fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c580adec0 0x7f3c580cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c567fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f3c580adec0 0x7f3c580cac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58530/0 (socket says 172.21.15.40:58530) 2023-10-16T08:38:53.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c567fc640 1 -- 172.21.15.40:0/1186044800 learned_addr learned my addr 172.21.15.40:0/1186044800 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c56ffd640 1 --2- 172.21.15.40:0/1186044800 >> v2:172.21.15.86:3300/0 conn(0x7f3c5815b040 0x7f3c580ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c567fc640 1 -- 172.21.15.40:0/1186044800 >> v2:172.21.15.86:3300/0 conn(0x7f3c5815b040 msgr2=0x7f3c580ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c567fc640 1 --2- 172.21.15.40:0/1186044800 >> v2:172.21.15.86:3300/0 conn(0x7f3c5815b040 0x7f3c580ca740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c567fc640 1 -- 172.21.15.40:0/1186044800 >> v1:172.21.15.86:6789/0 conn(0x7f3c580c8fb0 legacy=0x7f3c580c9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c567fc640 1 -- 172.21.15.40:0/1186044800 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c580cf5b0 con 0x7f3c580adec0 2023-10-16T08:38:53.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c56ffd640 1 --2- 172.21.15.40:0/1186044800 >> v2:172.21.15.86:3300/0 conn(0x7f3c5815b040 0x7f3c580ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:53.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c567fc640 1 --2- 172.21.15.40:0/1186044800 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c580adec0 0x7f3c580cac80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3c58069fe0 tx=0x7f3c4001c370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14924bdb4f1b96b4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:53.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c55ffb640 1 -- 172.21.15.40:0/1186044800 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c400ab600 con 0x7f3c580adec0 2023-10-16T08:38:53.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c55ffb640 1 -- 172.21.15.40:0/1186044800 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c400a7050 con 0x7f3c580adec0 2023-10-16T08:38:53.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c55ffb640 1 -- 172.21.15.40:0/1186044800 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c400ab980 con 0x7f3c580adec0 2023-10-16T08:38:53.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c5d965640 1 -- 172.21.15.40:0/1186044800 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c580adec0 msgr2=0x7f3c580cac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:53.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c5d965640 1 --2- 172.21.15.40:0/1186044800 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c580adec0 0x7f3c580cac80 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f3c58069fe0 tx=0x7f3c4001c370 comp rx=0 tx=0).stop 2023-10-16T08:38:53.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c5d965640 1 -- 172.21.15.40:0/1186044800 shutdown_connections 2023-10-16T08:38:53.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c5d965640 1 --2- 172.21.15.40:0/1186044800 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c580adec0 0x7f3c580cac80 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c5d965640 1 --2- 172.21.15.40:0/1186044800 >> v2:172.21.15.86:3300/0 conn(0x7f3c5815b040 0x7f3c580ca740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c5d965640 1 -- 172.21.15.40:0/1186044800 >> 172.21.15.40:0/1186044800 conn(0x7f3c580ce960 msgr2=0x7f3c580ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:53.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c5d965640 1 -- 172.21.15.40:0/1186044800 shutdown_connections 2023-10-16T08:38:53.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.820+0000 7f3c5d965640 1 -- 172.21.15.40:0/1186044800 wait complete. 2023-10-16T08:38:53.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c5d965640 1 Processor -- start 2023-10-16T08:38:53.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c5d965640 1 -- start start 2023-10-16T08:38:53.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c5d965640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c580adec0 0x7f3c580c8630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:53.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c5d965640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c580c8fb0 0x7f3c58159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:53.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c5d965640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c5815b040 0x7f3c580c3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:53.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c5d965640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c58155710 con 0x7f3c580adec0 2023-10-16T08:38:53.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c5d965640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c580cdae0 con 0x7f3c5815b040 2023-10-16T08:38:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c5d965640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c580cd520 con 0x7f3c580c8fb0 2023-10-16T08:38:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c56ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c580adec0 0x7f3c580c8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c56ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c580adec0 0x7f3c580c8630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58538/0 (socket says 172.21.15.40:58538) 2023-10-16T08:38:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c56ffd640 1 -- 172.21.15.40:0/3668142186 learned_addr learned my addr 172.21.15.40:0/3668142186 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:53.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c56ffd640 1 -- 172.21.15.40:0/3668142186 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c580c8fb0 msgr2=0x7f3c58159de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:53.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c567fc640 1 --2- 172.21.15.40:0/3668142186 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c580c8fb0 0x7f3c58159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:53.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c577fe640 1 --2- 172.21.15.40:0/3668142186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c5815b040 0x7f3c580c3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:53.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c56ffd640 1 --2- 172.21.15.40:0/3668142186 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c580c8fb0 0x7f3c58159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c56ffd640 1 -- 172.21.15.40:0/3668142186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c5815b040 msgr2=0x7f3c580c3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:53.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c56ffd640 1 --2- 172.21.15.40:0/3668142186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c5815b040 0x7f3c580c3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:53.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c56ffd640 1 -- 172.21.15.40:0/3668142186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c400094d0 con 0x7f3c580adec0 2023-10-16T08:38:53.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c56ffd640 1 --2- 172.21.15.40:0/3668142186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c580adec0 0x7f3c580c8630 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800cf60 tx=0x7f3c4800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:53.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c4ffff640 1 -- 172.21.15.40:0/3668142186 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4800d840 con 0x7f3c580adec0 2023-10-16T08:38:53.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c4ffff640 1 -- 172.21.15.40:0/3668142186 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c4800d9a0 con 0x7f3c580adec0 2023-10-16T08:38:53.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c5d965640 1 -- 172.21.15.40:0/3668142186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c580cd800 con 0x7f3c580adec0 2023-10-16T08:38:53.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c4ffff640 1 -- 172.21.15.40:0/3668142186 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c4800dd30 con 0x7f3c580adec0 2023-10-16T08:38:53.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c5d965640 1 -- 172.21.15.40:0/3668142186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c580c3860 con 0x7f3c580adec0 2023-10-16T08:38:53.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.824+0000 7f3c5d965640 1 -- 172.21.15.40:0/3668142186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c5804e9c0 con 0x7f3c580adec0 2023-10-16T08:38:53.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.828+0000 7f3c4ffff640 1 -- 172.21.15.40:0/3668142186 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3c48015440 con 0x7f3c580adec0 2023-10-16T08:38:53.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.828+0000 7f3c4ffff640 1 --2- 172.21.15.40:0/3668142186 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3c380664a0 0x7f3c38068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.828+0000 7f3c4ffff640 1 -- 172.21.15.40:0/3668142186 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c48084890 con 0x7f3c580adec0 2023-10-16T08:38:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.828+0000 7f3c567fc640 1 --2- 172.21.15.40:0/3668142186 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3c380664a0 0x7f3c38068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:53.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.828+0000 7f3c567fc640 1 --2- 172.21.15.40:0/3668142186 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3c380664a0 0x7f3c38068960 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3c400ad000 tx=0x7f3c40008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:53.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:53.832+0000 7f3c4ffff640 1 -- 172.21.15.40:0/3668142186 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f3c48051dc0 con 0x7f3c580adec0 2023-10-16T08:38:54.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.016+0000 7f3c5d965640 1 -- 172.21.15.40:0/3668142186 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f3c580cb170 con 0x7f3c580adec0 2023-10-16T08:38:54.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.016+0000 7f3c4ffff640 1 -- 172.21.15.40:0/3668142186 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f3c4801a080 con 0x7f3c580adec0 2023-10-16T08:38:54.022 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:54.022 INFO:teuthology.orchestra.run.smithi040.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-16T08:38:54.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.020+0000 7f3c5d965640 1 -- 172.21.15.40:0/3668142186 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3c380664a0 msgr2=0x7f3c38068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.020+0000 7f3c5d965640 1 --2- 172.21.15.40:0/3668142186 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3c380664a0 0x7f3c38068960 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f3c400ad000 tx=0x7f3c40008040 comp rx=0 tx=0).stop 2023-10-16T08:38:54.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.020+0000 7f3c5d965640 1 -- 172.21.15.40:0/3668142186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c580adec0 msgr2=0x7f3c580c8630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.020+0000 7f3c5d965640 1 --2- 172.21.15.40:0/3668142186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c580adec0 0x7f3c580c8630 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3c4800cf60 tx=0x7f3c4800cf90 comp rx=0 tx=0).stop 2023-10-16T08:38:54.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.020+0000 7f3c5d965640 1 -- 172.21.15.40:0/3668142186 shutdown_connections 2023-10-16T08:38:54.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.020+0000 7f3c5d965640 1 --2- 172.21.15.40:0/3668142186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c5815b040 0x7f3c580c3d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.020+0000 7f3c5d965640 1 --2- 172.21.15.40:0/3668142186 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c580c8fb0 0x7f3c58159de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.020+0000 7f3c5d965640 1 --2- 172.21.15.40:0/3668142186 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3c380664a0 0x7f3c38068960 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.020+0000 7f3c5d965640 1 --2- 172.21.15.40:0/3668142186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c580adec0 0x7f3c580c8630 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.020+0000 7f3c5d965640 1 -- 172.21.15.40:0/3668142186 >> 172.21.15.40:0/3668142186 conn(0x7f3c580ce960 msgr2=0x7f3c58158a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:54.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.020+0000 7f3c5d965640 1 -- 172.21.15.40:0/3668142186 shutdown_connections 2023-10-16T08:38:54.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.020+0000 7f3c5d965640 1 -- 172.21.15.40:0/3668142186 wait complete. 2023-10-16T08:38:54.039 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-16T08:38:54.039 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:38:54.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.152+0000 7fa41ddbf640 1 Processor -- start 2023-10-16T08:38:54.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.152+0000 7fa41ddbf640 1 -- start start 2023-10-16T08:38:54.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa41ddbf640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41815d180 0x7fa41806b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa41ddbf640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa4180ad820 0x7fa4180c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa41ddbf640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4180cf480 con 0x7fa41815d560 2023-10-16T08:38:54.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa41ddbf640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4180cf760 con 0x7fa4180ad820 2023-10-16T08:38:54.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa41ddbf640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4180c8fb0 con 0x7fa41815d180 2023-10-16T08:38:54.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa416ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa41815d560 0x7fa4180c9570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58436/0 (socket says 172.21.15.40:58436) 2023-10-16T08:38:54.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa416ffd640 1 -- 172.21.15.40:0/1768065709 learned_addr learned my addr 172.21.15.40:0/1768065709 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:54.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa4177fe640 1 --2- 172.21.15.40:0/1768065709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41815d180 0x7fa41806b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa417fff640 1 --2- 172.21.15.40:0/1768065709 >> v2:172.21.15.86:3300/0 conn(0x7fa4180ad820 0x7fa4180c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa4167fc640 1 -- 172.21.15.40:0/1768065709 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4075363261 0 0) 0x7fa4180cf480 con 0x7fa41815d560 2023-10-16T08:38:54.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa4167fc640 1 -- 172.21.15.40:0/1768065709 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa400003530 con 0x7fa41815d560 2023-10-16T08:38:54.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa4167fc640 1 -- 172.21.15.40:0/1768065709 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1365297423 0 0) 0x7fa400003530 con 0x7fa41815d560 2023-10-16T08:38:54.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa4167fc640 1 -- 172.21.15.40:0/1768065709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41815d180 msgr2=0x7fa41806b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:54.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa4167fc640 1 --2- 172.21.15.40:0/1768065709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41815d180 0x7fa41806b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa4167fc640 1 -- 172.21.15.40:0/1768065709 >> v2:172.21.15.86:3300/0 conn(0x7fa4180ad820 msgr2=0x7fa4180c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:54.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa4167fc640 1 --2- 172.21.15.40:0/1768065709 >> v2:172.21.15.86:3300/0 conn(0x7fa4180ad820 0x7fa4180c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa4167fc640 1 -- 172.21.15.40:0/1768065709 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4180c9290 con 0x7fa41815d560 2023-10-16T08:38:54.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa4167fc640 1 -- 172.21.15.40:0/1768065709 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7fa40c001680 con 0x7fa41815d560 2023-10-16T08:38:54.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa4167fc640 1 -- 172.21.15.40:0/1768065709 >> v1:172.21.15.40:6789/0 conn(0x7fa41815d560 legacy=0x7fa4180c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa4167fc640 1 --2- 172.21.15.40:0/1768065709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4000039f0 0x7fa400003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa4167fc640 1 --2- 172.21.15.40:0/1768065709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4000047d0 0x7fa400004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa4167fc640 1 --2- 172.21.15.40:0/1768065709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41815d180 0x7fa400005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa4167fc640 1 -- 172.21.15.40:0/1768065709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4180cf760 con 0x7fa4000047d0 2023-10-16T08:38:54.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa4167fc640 1 -- 172.21.15.40:0/1768065709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4180c8fb0 con 0x7fa4000039f0 2023-10-16T08:38:54.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa4167fc640 1 -- 172.21.15.40:0/1768065709 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa40000b990 con 0x7fa41815d180 2023-10-16T08:38:54.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa416ffd640 1 --2- 172.21.15.40:0/1768065709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4000047d0 0x7fa400004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa417fff640 1 --2- 172.21.15.40:0/1768065709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41815d180 0x7fa400005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa4177fe640 1 --2- 172.21.15.40:0/1768065709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4000039f0 0x7fa400003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa416ffd640 1 -- 172.21.15.40:0/1768065709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41815d180 msgr2=0x7fa400005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:54.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa416ffd640 1 --2- 172.21.15.40:0/1768065709 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41815d180 0x7fa400005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa416ffd640 1 -- 172.21.15.40:0/1768065709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4000039f0 msgr2=0x7fa400003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:54.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa416ffd640 1 --2- 172.21.15.40:0/1768065709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4000039f0 0x7fa400003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.156+0000 7fa416ffd640 1 -- 172.21.15.40:0/1768065709 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa400003530 con 0x7fa4000047d0 2023-10-16T08:38:54.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa416ffd640 1 --2- 172.21.15.40:0/1768065709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4000047d0 0x7fa400004c50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00c2c0 tx=0x7fa40c01f430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a925676188b98160 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:54.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa4167fc640 1 -- 172.21.15.40:0/1768065709 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0a8cd0 con 0x7fa4000047d0 2023-10-16T08:38:54.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa4167fc640 1 -- 172.21.15.40:0/1768065709 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40c0a8e30 con 0x7fa4000047d0 2023-10-16T08:38:54.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa4167fc640 1 -- 172.21.15.40:0/1768065709 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c005b20 con 0x7fa4000047d0 2023-10-16T08:38:54.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa41ddbf640 1 -- 172.21.15.40:0/1768065709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4000047d0 msgr2=0x7fa400004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa41ddbf640 1 --2- 172.21.15.40:0/1768065709 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4000047d0 0x7fa400004c50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa40c00c2c0 tx=0x7fa40c01f430 comp rx=0 tx=0).stop 2023-10-16T08:38:54.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa417fff640 1 -- 172.21.15.40:0/1768065709 reap_dead start 2023-10-16T08:38:54.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa41ddbf640 1 -- 172.21.15.40:0/1768065709 shutdown_connections 2023-10-16T08:38:54.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa41ddbf640 1 -- 172.21.15.40:0/1768065709 >> 172.21.15.40:0/1768065709 conn(0x7fa4180ce960 msgr2=0x7fa4180ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:54.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa41ddbf640 1 -- 172.21.15.40:0/1768065709 shutdown_connections 2023-10-16T08:38:54.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa41ddbf640 1 -- 172.21.15.40:0/1768065709 wait complete. 2023-10-16T08:38:54.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa41ddbf640 1 Processor -- start 2023-10-16T08:38:54.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa41ddbf640 1 -- start start 2023-10-16T08:38:54.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa41ddbf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4000047d0 0x7fa4180c4d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa41ddbf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41815b260 0x7fa4181e7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa41ddbf640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa418070600 0x7fa418070a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa41ddbf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41815b6e0 con 0x7fa41815b260 2023-10-16T08:38:54.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa41ddbf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa418070fc0 con 0x7fa4000047d0 2023-10-16T08:38:54.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa41ddbf640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4180c3580 con 0x7fa418070600 2023-10-16T08:38:54.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa416ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41815b260 0x7fa4181e7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa416ffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41815b260 0x7fa4181e7080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58556/0 (socket says 172.21.15.40:58556) 2023-10-16T08:38:54.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa416ffd640 1 -- 172.21.15.40:0/4055258680 learned_addr learned my addr 172.21.15.40:0/4055258680 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:54.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa417fff640 1 --2- 172.21.15.40:0/4055258680 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa418070600 0x7fa418070a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa4177fe640 1 --2- 172.21.15.40:0/4055258680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4000047d0 0x7fa4180c4d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa416ffd640 1 -- 172.21.15.40:0/4055258680 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa418070600 msgr2=0x7fa418070a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa416ffd640 1 --2- 172.21.15.40:0/4055258680 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa418070600 0x7fa418070a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa416ffd640 1 -- 172.21.15.40:0/4055258680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4000047d0 msgr2=0x7fa4180c4d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa416ffd640 1 --2- 172.21.15.40:0/4055258680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4000047d0 0x7fa4180c4d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa416ffd640 1 -- 172.21.15.40:0/4055258680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4180c3860 con 0x7fa41815b260 2023-10-16T08:38:54.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa416ffd640 1 --2- 172.21.15.40:0/4055258680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41815b260 0x7fa4181e7080 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa40c007040 tx=0x7fa400005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:54.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa3fffff640 1 -- 172.21.15.40:0/4055258680 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c004ab0 con 0x7fa41815b260 2023-10-16T08:38:54.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.160+0000 7fa3fffff640 1 -- 172.21.15.40:0/4055258680 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa40c004c70 con 0x7fa41815b260 2023-10-16T08:38:54.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.164+0000 7fa41ddbf640 1 -- 172.21.15.40:0/4055258680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4180c3b40 con 0x7fa41815b260 2023-10-16T08:38:54.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.164+0000 7fa3fffff640 1 -- 172.21.15.40:0/4055258680 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa40c0070d0 con 0x7fa41815b260 2023-10-16T08:38:54.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.164+0000 7fa41ddbf640 1 -- 172.21.15.40:0/4055258680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4180c4100 con 0x7fa41815b260 2023-10-16T08:38:54.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.164+0000 7fa41ddbf640 1 -- 172.21.15.40:0/4055258680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa41815ca00 con 0x7fa41815b260 2023-10-16T08:38:54.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.164+0000 7fa3fffff640 1 -- 172.21.15.40:0/4055258680 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa40c0b8070 con 0x7fa41815b260 2023-10-16T08:38:54.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.164+0000 7fa3fffff640 1 --2- 172.21.15.40:0/4055258680 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa4040664a0 0x7fa404068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.164+0000 7fa3fffff640 1 -- 172.21.15.40:0/4055258680 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa40c121290 con 0x7fa41815b260 2023-10-16T08:38:54.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.164+0000 7fa4177fe640 1 --2- 172.21.15.40:0/4055258680 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa4040664a0 0x7fa404068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.164+0000 7fa4177fe640 1 --2- 172.21.15.40:0/4055258680 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa4040664a0 0x7fa404068960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa418071560 tx=0x7fa408008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:54.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.168+0000 7fa3fffff640 1 -- 172.21.15.40:0/4055258680 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fa40c0ee9a0 con 0x7fa41815b260 2023-10-16T08:38:54.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.320+0000 7fa41ddbf640 1 -- 172.21.15.40:0/4055258680 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa41804e9c0 con 0x7fa41815b260 2023-10-16T08:38:54.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.320+0000 7fa3fffff640 1 -- 172.21.15.40:0/4055258680 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16445 (secure 0 0 0) 0x7fa40c0f1fc0 con 0x7fa41815b260 2023-10-16T08:38:54.327 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:54.327 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:46.611098+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:38:54.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.324+0000 7fa41ddbf640 1 -- 172.21.15.40:0/4055258680 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa4040664a0 msgr2=0x7fa404068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.324+0000 7fa41ddbf640 1 --2- 172.21.15.40:0/4055258680 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa4040664a0 0x7fa404068960 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa418071560 tx=0x7fa408008040 comp rx=0 tx=0).stop 2023-10-16T08:38:54.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.324+0000 7fa41ddbf640 1 -- 172.21.15.40:0/4055258680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41815b260 msgr2=0x7fa4181e7080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.324+0000 7fa41ddbf640 1 --2- 172.21.15.40:0/4055258680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41815b260 0x7fa4181e7080 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fa40c007040 tx=0x7fa400005800 comp rx=0 tx=0).stop 2023-10-16T08:38:54.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.324+0000 7fa41ddbf640 1 -- 172.21.15.40:0/4055258680 shutdown_connections 2023-10-16T08:38:54.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.324+0000 7fa41ddbf640 1 --2- 172.21.15.40:0/4055258680 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa418070600 0x7fa418070a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.324+0000 7fa41ddbf640 1 --2- 172.21.15.40:0/4055258680 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa4040664a0 0x7fa404068960 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.324+0000 7fa41ddbf640 1 --2- 172.21.15.40:0/4055258680 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41815b260 0x7fa4181e7080 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.324+0000 7fa41ddbf640 1 --2- 172.21.15.40:0/4055258680 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa4000047d0 0x7fa4180c4d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.324+0000 7fa41ddbf640 1 -- 172.21.15.40:0/4055258680 >> 172.21.15.40:0/4055258680 conn(0x7fa4180ce960 msgr2=0x7fa41815c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:54.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.324+0000 7fa41ddbf640 1 -- 172.21.15.40:0/4055258680 shutdown_connections 2023-10-16T08:38:54.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.324+0000 7fa41ddbf640 1 -- 172.21.15.40:0/4055258680 wait complete. 2023-10-16T08:38:54.344 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:38:33.014668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:38:43.055215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:38:43.971666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:38:54.345 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:38:54.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57fb98f640 1 Processor -- start 2023-10-16T08:38:54.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57fb98f640 1 -- start start 2023-10-16T08:38:54.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57fb98f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f406bc60 0x7f57f40c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57fb98f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f40cc460 con 0x7f57f40cceb0 2023-10-16T08:38:54.465 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57fb98f640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f40cca20 con 0x7f57f415d970 2023-10-16T08:38:54.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57fb98f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40c9420 con 0x7f57f406bc60 2023-10-16T08:38:54.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57f9f2c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f57f40cceb0 0x7f57f40cd340 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:58448/0 (socket says 172.21.15.40:58448) 2023-10-16T08:38:54.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57f9f2c640 1 -- 172.21.15.40:0/1059146101 learned_addr learned my addr 172.21.15.40:0/1059146101 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:54.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57f972b640 1 --2- 172.21.15.40:0/1059146101 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f406bc60 0x7f57f40c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57ebfff640 1 -- 172.21.15.40:0/1059146101 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2719013078 0 0) 0x7f57f40cc460 con 0x7f57f40cceb0 2023-10-16T08:38:54.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57ebfff640 1 -- 172.21.15.40:0/1059146101 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e0003530 con 0x7f57f40cceb0 2023-10-16T08:38:54.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57ebfff640 1 -- 172.21.15.40:0/1059146101 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1359138705 0 0) 0x7f57e0003530 con 0x7f57f40cceb0 2023-10-16T08:38:54.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57ebfff640 1 -- 172.21.15.40:0/1059146101 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f406bc60 msgr2=0x7f57f40c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:54.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57ebfff640 1 --2- 172.21.15.40:0/1059146101 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f406bc60 0x7f57f40c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57ebfff640 1 -- 172.21.15.40:0/1059146101 >> v1:172.21.15.86:6789/0 conn(0x7f57f415d970 legacy=0x7f57f406c040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57ebfff640 1 -- 172.21.15.40:0/1059146101 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f40c9630 con 0x7f57f40cceb0 2023-10-16T08:38:54.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57ebfff640 1 -- 172.21.15.40:0/1059146101 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f57ec001a00 con 0x7f57f40cceb0 2023-10-16T08:38:54.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57ebfff640 1 -- 172.21.15.40:0/1059146101 >> v1:172.21.15.40:6789/0 conn(0x7f57f40cceb0 legacy=0x7f57f40cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57ebfff640 1 --2- 172.21.15.40:0/1059146101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e0003a20 0x7f57e0003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.468 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57ebfff640 1 --2- 172.21.15.40:0/1059146101 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f406bc60 0x7f57e00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57ebfff640 1 --2- 172.21.15.40:0/1059146101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e0004d10 0x7f57e0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57ebfff640 1 -- 172.21.15.40:0/1059146101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40c9420 con 0x7f57e0003a20 2023-10-16T08:38:54.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57ebfff640 1 -- 172.21.15.40:0/1059146101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e000bb20 con 0x7f57e0004d10 2023-10-16T08:38:54.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57ebfff640 1 -- 172.21.15.40:0/1059146101 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e000bcb0 con 0x7f57f406bc60 2023-10-16T08:38:54.469 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57f972b640 1 --2- 172.21.15.40:0/1059146101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e0003a20 0x7f57e0003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57f8f2a640 1 --2- 172.21.15.40:0/1059146101 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f406bc60 0x7f57e00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57f9f2c640 1 --2- 172.21.15.40:0/1059146101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e0004d10 0x7f57e0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57f972b640 1 -- 172.21.15.40:0/1059146101 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f406bc60 msgr2=0x7f57e00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:54.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57f972b640 1 --2- 172.21.15.40:0/1059146101 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f406bc60 0x7f57e00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57f972b640 1 -- 172.21.15.40:0/1059146101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e0004d10 msgr2=0x7f57e0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:54.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57f972b640 1 --2- 172.21.15.40:0/1059146101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e0004d10 0x7f57e0005190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.456+0000 7f57f972b640 1 -- 172.21.15.40:0/1059146101 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e0003530 con 0x7f57e0003a20 2023-10-16T08:38:54.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.460+0000 7f57f972b640 1 --2- 172.21.15.40:0/1059146101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e0003a20 0x7f57e0003e40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f57e400aae0 tx=0x7f57e401df30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb15cc181bc10a36 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:54.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.460+0000 7f57ebfff640 1 -- 172.21.15.40:0/1059146101 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e40a6da0 con 0x7f57e0003a20 2023-10-16T08:38:54.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.460+0000 7f57ebfff640 1 -- 172.21.15.40:0/1059146101 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57e40ae440 con 0x7f57e0003a20 2023-10-16T08:38:54.471 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.460+0000 7f57ebfff640 1 -- 172.21.15.40:0/1059146101 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57e40ae7f0 con 0x7f57e0003a20 2023-10-16T08:38:54.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.460+0000 7f57fb98f640 1 -- 172.21.15.40:0/1059146101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e0003a20 msgr2=0x7f57e0003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.460+0000 7f57fb98f640 1 --2- 172.21.15.40:0/1059146101 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e0003a20 0x7f57e0003e40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f57e400aae0 tx=0x7f57e401df30 comp rx=0 tx=0).stop 2023-10-16T08:38:54.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.460+0000 7f57f9f2c640 1 -- 172.21.15.40:0/1059146101 reap_dead start 2023-10-16T08:38:54.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.460+0000 7f57fb98f640 1 -- 172.21.15.40:0/1059146101 shutdown_connections 2023-10-16T08:38:54.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.460+0000 7f57fb98f640 1 -- 172.21.15.40:0/1059146101 >> 172.21.15.40:0/1059146101 conn(0x7f57f40ce890 msgr2=0x7f57f40ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:54.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.460+0000 7f57fb98f640 1 -- 172.21.15.40:0/1059146101 shutdown_connections 2023-10-16T08:38:54.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.460+0000 7f57fb98f640 1 -- 172.21.15.40:0/1059146101 wait complete. 2023-10-16T08:38:54.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.460+0000 7f57fb98f640 1 Processor -- start 2023-10-16T08:38:54.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.460+0000 7f57fb98f640 1 -- start start 2023-10-16T08:38:54.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57fb98f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e0003a20 0x7f57f414bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57fb98f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f41581b0 0x7f57f40701d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57fb98f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f41f3b30 0x7f57f41f3fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57fb98f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f4158610 con 0x7f57f41f3b30 2023-10-16T08:38:54.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57fb98f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f40c4ad0 con 0x7f57e0003a20 2023-10-16T08:38:54.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57fb98f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f414c5d0 con 0x7f57f41581b0 2023-10-16T08:38:54.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57f9f2c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f41f3b30 0x7f57f41f3fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57f9f2c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f41f3b30 0x7f57f41f3fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:58576/0 (socket says 172.21.15.40:58576) 2023-10-16T08:38:54.477 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57f9f2c640 1 -- 172.21.15.40:0/2356370050 learned_addr learned my addr 172.21.15.40:0/2356370050 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:54.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57f972b640 1 --2- 172.21.15.40:0/2356370050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e0003a20 0x7f57f414bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57f8f2a640 1 --2- 172.21.15.40:0/2356370050 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f41581b0 0x7f57f40701d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57f9f2c640 1 -- 172.21.15.40:0/2356370050 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f41581b0 msgr2=0x7f57f40701d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57f9f2c640 1 --2- 172.21.15.40:0/2356370050 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f41581b0 0x7f57f40701d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57f9f2c640 1 -- 172.21.15.40:0/2356370050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e0003a20 msgr2=0x7f57f414bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57f9f2c640 1 --2- 172.21.15.40:0/2356370050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e0003a20 0x7f57f414bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57f9f2c640 1 -- 172.21.15.40:0/2356370050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f414c8b0 con 0x7f57f41f3b30 2023-10-16T08:38:54.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57f9f2c640 1 --2- 172.21.15.40:0/2356370050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f41f3b30 0x7f57f41f3fb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f57f40cb2e0 tx=0x7f57ec00b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:54.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57e9ffb640 1 -- 172.21.15.40:0/2356370050 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec00b8d0 con 0x7f57f41f3b30 2023-10-16T08:38:54.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57e9ffb640 1 -- 172.21.15.40:0/2356370050 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57ec00ba30 con 0x7f57f41f3b30 2023-10-16T08:38:54.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57fb98f640 1 -- 172.21.15.40:0/2356370050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f414cb90 con 0x7f57f41f3b30 2023-10-16T08:38:54.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57e9ffb640 1 -- 172.21.15.40:0/2356370050 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57ec00bde0 con 0x7f57f41f3b30 2023-10-16T08:38:54.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57fb98f640 1 -- 172.21.15.40:0/2356370050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f414d150 con 0x7f57f41f3b30 2023-10-16T08:38:54.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57fb98f640 1 -- 172.21.15.40:0/2356370050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f4067a20 con 0x7f57f41f3b30 2023-10-16T08:38:54.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57e9ffb640 1 -- 172.21.15.40:0/2356370050 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f57ec007110 con 0x7f57f41f3b30 2023-10-16T08:38:54.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57e9ffb640 1 --2- 172.21.15.40:0/2356370050 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f57d80664d0 0x7f57d8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57e9ffb640 1 -- 172.21.15.40:0/2356370050 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57ec083c50 con 0x7f57f41f3b30 2023-10-16T08:38:54.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.464+0000 7f57f972b640 1 --2- 172.21.15.40:0/2356370050 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f57d80664d0 0x7f57d8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.468+0000 7f57f972b640 1 --2- 172.21.15.40:0/2356370050 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f57d80664d0 0x7f57d8068990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f57e40b0000 tx=0x7f57e4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:54.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.468+0000 7f57e9ffb640 1 -- 172.21.15.40:0/2356370050 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f57ec0510d0 con 0x7f57f41f3b30 2023-10-16T08:38:54.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.620+0000 7f57fb98f640 1 -- 172.21.15.40:0/2356370050 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f57f404e9c0 con 0x7f57f41f3b30 2023-10-16T08:38:54.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.620+0000 7f57e9ffb640 1 -- 172.21.15.40:0/2356370050 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57ec0546f0 con 0x7f57f41f3b30 2023-10-16T08:38:54.626 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-16T08:38:54.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.624+0000 7f57fb98f640 1 -- 172.21.15.40:0/2356370050 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f57d80664d0 msgr2=0x7f57d8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.624+0000 7f57fb98f640 1 --2- 172.21.15.40:0/2356370050 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f57d80664d0 0x7f57d8068990 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f57e40b0000 tx=0x7f57e4008040 comp rx=0 tx=0).stop 2023-10-16T08:38:54.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.624+0000 7f57fb98f640 1 -- 172.21.15.40:0/2356370050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f41f3b30 msgr2=0x7f57f41f3fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.624+0000 7f57fb98f640 1 --2- 172.21.15.40:0/2356370050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f41f3b30 0x7f57f41f3fb0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f57f40cb2e0 tx=0x7f57ec00b0d0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.624+0000 7f57fb98f640 1 -- 172.21.15.40:0/2356370050 shutdown_connections 2023-10-16T08:38:54.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.624+0000 7f57fb98f640 1 --2- 172.21.15.40:0/2356370050 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f57d80664d0 0x7f57d8068990 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.624+0000 7f57fb98f640 1 --2- 172.21.15.40:0/2356370050 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f41f3b30 0x7f57f41f3fb0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.624+0000 7f57fb98f640 1 --2- 172.21.15.40:0/2356370050 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f41581b0 0x7f57f40701d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.624+0000 7f57fb98f640 1 --2- 172.21.15.40:0/2356370050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57e0003a20 0x7f57f414bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.624+0000 7f57fb98f640 1 -- 172.21.15.40:0/2356370050 >> 172.21.15.40:0/2356370050 conn(0x7f57f40ce890 msgr2=0x7f57f4158df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:54.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.624+0000 7f57fb98f640 1 -- 172.21.15.40:0/2356370050 shutdown_connections 2023-10-16T08:38:54.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.624+0000 7f57fb98f640 1 -- 172.21.15.40:0/2356370050 wait complete. 2023-10-16T08:38:54.643 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:38:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.752+0000 7fd12fbb9640 1 Processor -- start 2023-10-16T08:38:54.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.752+0000 7fd12fbb9640 1 -- start start 2023-10-16T08:38:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12fbb9640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd1280adb70 0x7fd1280c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12fbb9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd12806d690 con 0x7fd12806f370 2023-10-16T08:38:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12fbb9640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1280c1640 con 0x7fd1280c1c00 2023-10-16T08:38:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12fbb9640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1280c1900 con 0x7fd1280adb70 2023-10-16T08:38:54.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12e156640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd12806f370 0x7fd1280ca6c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50388/0 (socket says 172.21.15.40:50388) 2023-10-16T08:38:54.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12e156640 1 -- 172.21.15.40:0/149036617 learned_addr learned my addr 172.21.15.40:0/149036617 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:54.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12d955640 1 --2- 172.21.15.40:0/149036617 >> v2:172.21.15.86:3300/0 conn(0x7fd1280adb70 0x7fd1280c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12c953640 1 -- 172.21.15.40:0/149036617 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 451118848 0 0) 0x7fd12806d690 con 0x7fd12806f370 2023-10-16T08:38:54.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12c953640 1 -- 172.21.15.40:0/149036617 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd110003530 con 0x7fd12806f370 2023-10-16T08:38:54.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12c953640 1 -- 172.21.15.40:0/149036617 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1981245472 0 0) 0x7fd110003530 con 0x7fd12806f370 2023-10-16T08:38:54.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12c953640 1 -- 172.21.15.40:0/149036617 >> v2:172.21.15.86:3300/0 conn(0x7fd1280adb70 msgr2=0x7fd1280c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:54.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12c953640 1 --2- 172.21.15.40:0/149036617 >> v2:172.21.15.86:3300/0 conn(0x7fd1280adb70 0x7fd1280c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12c953640 1 -- 172.21.15.40:0/149036617 >> v1:172.21.15.86:6789/0 conn(0x7fd1280c1c00 legacy=0x7fd1280cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12c953640 1 -- 172.21.15.40:0/149036617 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1280cc460 con 0x7fd12806f370 2023-10-16T08:38:54.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12c953640 1 -- 172.21.15.40:0/149036617 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7fd118001680 con 0x7fd12806f370 2023-10-16T08:38:54.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12c953640 1 -- 172.21.15.40:0/149036617 >> v1:172.21.15.40:6789/0 conn(0x7fd12806f370 legacy=0x7fd1280ca6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12c953640 1 --2- 172.21.15.40:0/149036617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1100039f0 0x7fd110003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12c953640 1 --2- 172.21.15.40:0/149036617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1100047d0 0x7fd110004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12c953640 1 --2- 172.21.15.40:0/149036617 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd110005190 0x7fd11000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12c953640 1 -- 172.21.15.40:0/149036617 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1280c1900 con 0x7fd1100047d0 2023-10-16T08:38:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12c953640 1 -- 172.21.15.40:0/149036617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11000c0c0 con 0x7fd1100039f0 2023-10-16T08:38:54.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12c953640 1 -- 172.21.15.40:0/149036617 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd11000c280 con 0x7fd110005190 2023-10-16T08:38:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12d154640 1 --2- 172.21.15.40:0/149036617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1100047d0 0x7fd110004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12d955640 1 --2- 172.21.15.40:0/149036617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1100039f0 0x7fd110003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12e156640 1 --2- 172.21.15.40:0/149036617 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd110005190 0x7fd11000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12d154640 1 -- 172.21.15.40:0/149036617 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd110005190 msgr2=0x7fd11000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:54.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12d154640 1 --2- 172.21.15.40:0/149036617 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd110005190 0x7fd11000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12d154640 1 -- 172.21.15.40:0/149036617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1100039f0 msgr2=0x7fd110003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12d154640 1 --2- 172.21.15.40:0/149036617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1100039f0 0x7fd110003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12d154640 1 -- 172.21.15.40:0/149036617 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd110003530 con 0x7fd1100047d0 2023-10-16T08:38:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12d955640 1 --2- 172.21.15.40:0/149036617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1100039f0 0x7fd110003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:38:54.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12e156640 1 --2- 172.21.15.40:0/149036617 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd110005190 0x7fd11000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:54.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.756+0000 7fd12e156640 1 -- 172.21.15.40:0/149036617 reap_dead start 2023-10-16T08:38:54.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12d154640 1 --2- 172.21.15.40:0/149036617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1100047d0 0x7fd110004c50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd124001e70 tx=0x7fd12401c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6a098ccb8a07ee7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:54.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12c953640 1 -- 172.21.15.40:0/149036617 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124003b80 con 0x7fd1100047d0 2023-10-16T08:38:54.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12c953640 1 -- 172.21.15.40:0/149036617 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd118001d80 con 0x7fd1100047d0 2023-10-16T08:38:54.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12c953640 1 -- 172.21.15.40:0/149036617 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd124005080 con 0x7fd1100047d0 2023-10-16T08:38:54.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12fbb9640 1 -- 172.21.15.40:0/149036617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1100047d0 msgr2=0x7fd110004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12fbb9640 1 --2- 172.21.15.40:0/149036617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1100047d0 0x7fd110004c50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fd124001e70 tx=0x7fd12401c740 comp rx=0 tx=0).stop 2023-10-16T08:38:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12fbb9640 1 -- 172.21.15.40:0/149036617 shutdown_connections 2023-10-16T08:38:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12fbb9640 1 --2- 172.21.15.40:0/149036617 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd1100047d0 0x7fd110004c50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12fbb9640 1 -- 172.21.15.40:0/149036617 >> 172.21.15.40:0/149036617 conn(0x7fd1280ce890 msgr2=0x7fd1280ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12fbb9640 1 -- 172.21.15.40:0/149036617 shutdown_connections 2023-10-16T08:38:54.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12fbb9640 1 -- 172.21.15.40:0/149036617 wait complete. 2023-10-16T08:38:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12fbb9640 1 Processor -- start 2023-10-16T08:38:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12fbb9640 1 -- start start 2023-10-16T08:38:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12fbb9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1100047d0 0x7fd1280c63b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12fbb9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1280bd450 0x7fd128162700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12fbb9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd12815db40 0x7fd12815dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12fbb9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1280c5960 con 0x7fd12815db40 2023-10-16T08:38:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12fbb9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd12815e500 con 0x7fd1100047d0 2023-10-16T08:38:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12fbb9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1280c88c0 con 0x7fd1280bd450 2023-10-16T08:38:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12e156640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd12815db40 0x7fd12815dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12e156640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd12815db40 0x7fd12815dfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:33996/0 (socket says 172.21.15.40:33996) 2023-10-16T08:38:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12e156640 1 -- 172.21.15.40:0/3068041731 learned_addr learned my addr 172.21.15.40:0/3068041731 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:54.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12d154640 1 --2- 172.21.15.40:0/3068041731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1280bd450 0x7fd128162700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12d955640 1 --2- 172.21.15.40:0/3068041731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1100047d0 0x7fd1280c63b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12e156640 1 -- 172.21.15.40:0/3068041731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1280bd450 msgr2=0x7fd128162700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12e156640 1 --2- 172.21.15.40:0/3068041731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1280bd450 0x7fd128162700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12e156640 1 -- 172.21.15.40:0/3068041731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1100047d0 msgr2=0x7fd1280c63b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12e156640 1 --2- 172.21.15.40:0/3068041731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1100047d0 0x7fd1280c63b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12e156640 1 -- 172.21.15.40:0/3068041731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1280c8ba0 con 0x7fd12815db40 2023-10-16T08:38:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12d154640 1 --2- 172.21.15.40:0/3068041731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1280bd450 0x7fd128162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12d955640 1 --2- 172.21.15.40:0/3068041731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1100047d0 0x7fd1280c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12e156640 1 --2- 172.21.15.40:0/3068041731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd12815db40 0x7fd12815dfc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd12806ad00 tx=0x7fd118010bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd11e7fc640 1 -- 172.21.15.40:0/3068041731 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11800e830 con 0x7fd12815db40 2023-10-16T08:38:54.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd11e7fc640 1 -- 172.21.15.40:0/3068041731 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd11800e990 con 0x7fd12815db40 2023-10-16T08:38:54.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12fbb9640 1 -- 172.21.15.40:0/3068041731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1280c8e80 con 0x7fd12815db40 2023-10-16T08:38:54.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd11e7fc640 1 -- 172.21.15.40:0/3068041731 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd11800ed20 con 0x7fd12815db40 2023-10-16T08:38:54.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.760+0000 7fd12fbb9640 1 -- 172.21.15.40:0/3068041731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1280c5e90 con 0x7fd12815db40 2023-10-16T08:38:54.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.764+0000 7fd12fbb9640 1 -- 172.21.15.40:0/3068041731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd128067a20 con 0x7fd12815db40 2023-10-16T08:38:54.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.764+0000 7fd11e7fc640 1 -- 172.21.15.40:0/3068041731 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd118015400 con 0x7fd12815db40 2023-10-16T08:38:54.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.764+0000 7fd11e7fc640 1 --2- 172.21.15.40:0/3068041731 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd1140664a0 0x7fd114068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:54.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.764+0000 7fd11e7fc640 1 -- 172.21.15.40:0/3068041731 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd1180868a0 con 0x7fd12815db40 2023-10-16T08:38:54.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.764+0000 7fd12d955640 1 --2- 172.21.15.40:0/3068041731 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd1140664a0 0x7fd114068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:54.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.764+0000 7fd12d955640 1 --2- 172.21.15.40:0/3068041731 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd1140664a0 0x7fd114068960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd120009b20 tx=0x7fd120008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:54.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.768+0000 7fd11e7fc640 1 -- 172.21.15.40:0/3068041731 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fd1180577b0 con 0x7fd12815db40 2023-10-16T08:38:54.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.916+0000 7fd12fbb9640 1 -- 172.21.15.40:0/3068041731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd12804e9c0 con 0x7fd12815db40 2023-10-16T08:38:54.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.920+0000 7fd11e7fc640 1 -- 172.21.15.40:0/3068041731 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd11801b020 con 0x7fd12815db40 2023-10-16T08:38:54.923 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-16T08:38:54.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.920+0000 7fd12fbb9640 1 -- 172.21.15.40:0/3068041731 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd1140664a0 msgr2=0x7fd114068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.920+0000 7fd12fbb9640 1 --2- 172.21.15.40:0/3068041731 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd1140664a0 0x7fd114068960 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fd120009b20 tx=0x7fd120008040 comp rx=0 tx=0).stop 2023-10-16T08:38:54.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.920+0000 7fd12fbb9640 1 -- 172.21.15.40:0/3068041731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd12815db40 msgr2=0x7fd12815dfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:54.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.920+0000 7fd12fbb9640 1 --2- 172.21.15.40:0/3068041731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd12815db40 0x7fd12815dfc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fd12806ad00 tx=0x7fd118010bb0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.920+0000 7fd12fbb9640 1 -- 172.21.15.40:0/3068041731 shutdown_connections 2023-10-16T08:38:54.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.920+0000 7fd12fbb9640 1 --2- 172.21.15.40:0/3068041731 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd1140664a0 0x7fd114068960 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.920+0000 7fd12fbb9640 1 --2- 172.21.15.40:0/3068041731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd12815db40 0x7fd12815dfc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.920+0000 7fd12fbb9640 1 --2- 172.21.15.40:0/3068041731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd1280bd450 0x7fd128162700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.920+0000 7fd12fbb9640 1 --2- 172.21.15.40:0/3068041731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1100047d0 0x7fd1280c63b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:54.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.920+0000 7fd12fbb9640 1 -- 172.21.15.40:0/3068041731 >> 172.21.15.40:0/3068041731 conn(0x7fd1280ce890 msgr2=0x7fd128161cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:54.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.920+0000 7fd12fbb9640 1 -- 172.21.15.40:0/3068041731 shutdown_connections 2023-10-16T08:38:54.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:54.920+0000 7fd12fbb9640 1 -- 172.21.15.40:0/3068041731 wait complete. 2023-10-16T08:38:54.941 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:38:55.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.064+0000 7fd6d21c0640 1 Processor -- start 2023-10-16T08:38:55.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.064+0000 7fd6d21c0640 1 -- start start 2023-10-16T08:38:55.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.064+0000 7fd6d21c0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd6cc0adec0 0x7fd6cc0c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:55.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.064+0000 7fd6d21c0640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6cc15b490 con 0x7fd6cc06b800 2023-10-16T08:38:55.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.064+0000 7fd6d21c0640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6cc06d230 con 0x7fd6cc159dd0 2023-10-16T08:38:55.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.064+0000 7fd6d21c0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6cc06cd70 con 0x7fd6cc0adec0 2023-10-16T08:38:55.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.064+0000 7fd6cbfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd6cc06b800 0x7fd6cc0cc460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50396/0 (socket says 172.21.15.40:50396) 2023-10-16T08:38:55.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.064+0000 7fd6caffd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd6cc0adec0 0x7fd6cc0c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:55.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.064+0000 7fd6cbfff640 1 -- 172.21.15.40:0/3337064809 learned_addr learned my addr 172.21.15.40:0/3337064809 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:55.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.064+0000 7fd6caffd640 1 -- 172.21.15.40:0/3337064809 >> v1:172.21.15.86:6789/0 conn(0x7fd6cc159dd0 legacy=0x7fd6cc0c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:55.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.064+0000 7fd6ca7fc640 1 -- 172.21.15.40:0/3337064809 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2479024500 0 0) 0x7fd6cc15b490 con 0x7fd6cc06b800 2023-10-16T08:38:55.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.064+0000 7fd6caffd640 1 -- 172.21.15.40:0/3337064809 >> v1:172.21.15.40:6789/0 conn(0x7fd6cc06b800 legacy=0x7fd6cc0cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:55.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.064+0000 7fd6caffd640 1 -- 172.21.15.40:0/3337064809 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6cc06cf50 con 0x7fd6cc0adec0 2023-10-16T08:38:55.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.068+0000 7fd6caffd640 1 --2- 172.21.15.40:0/3337064809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6cc0adec0 0x7fd6cc0c97f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00097c0 tx=0x7fd6c001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4531c20276e4e00 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:55.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.068+0000 7fd6ca7fc640 1 -- 172.21.15.40:0/3337064809 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c00a8050 con 0x7fd6cc0adec0 2023-10-16T08:38:55.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.068+0000 7fd6ca7fc640 1 -- 172.21.15.40:0/3337064809 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6c00ac5f0 con 0x7fd6cc0adec0 2023-10-16T08:38:55.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.068+0000 7fd6ca7fc640 1 -- 172.21.15.40:0/3337064809 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6c00ac980 con 0x7fd6cc0adec0 2023-10-16T08:38:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.068+0000 7fd6d21c0640 1 -- 172.21.15.40:0/3337064809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6cc0adec0 msgr2=0x7fd6cc0c97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.068+0000 7fd6d21c0640 1 --2- 172.21.15.40:0/3337064809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6cc0adec0 0x7fd6cc0c97f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00097c0 tx=0x7fd6c001c480 comp rx=0 tx=0).stop 2023-10-16T08:38:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.068+0000 7fd6d21c0640 1 -- 172.21.15.40:0/3337064809 shutdown_connections 2023-10-16T08:38:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.068+0000 7fd6d21c0640 1 --2- 172.21.15.40:0/3337064809 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6cc0adec0 0x7fd6cc0c97f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.068+0000 7fd6d21c0640 1 -- 172.21.15.40:0/3337064809 >> 172.21.15.40:0/3337064809 conn(0x7fd6cc0ce890 msgr2=0x7fd6cc0ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:55.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.068+0000 7fd6d21c0640 1 -- 172.21.15.40:0/3337064809 shutdown_connections 2023-10-16T08:38:55.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.068+0000 7fd6d21c0640 1 -- 172.21.15.40:0/3337064809 wait complete. 2023-10-16T08:38:55.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6d21c0640 1 Processor -- start 2023-10-16T08:38:55.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6d21c0640 1 -- start start 2023-10-16T08:38:55.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6d21c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6cc06b800 0x7fd6cc150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6d21c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6cc0adec0 0x7fd6cc0ac3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6d21c0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6cc159dd0 0x7fd6cc0aa1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6d21c0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc0abad0 con 0x7fd6cc06b800 2023-10-16T08:38:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6d21c0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc0aadc0 con 0x7fd6cc0adec0 2023-10-16T08:38:55.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6d21c0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6cc157500 con 0x7fd6cc159dd0 2023-10-16T08:38:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6cb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6cc06b800 0x7fd6cc150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6cb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6cc06b800 0x7fd6cc150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34020/0 (socket says 172.21.15.40:34020) 2023-10-16T08:38:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6cb7fe640 1 -- 172.21.15.40:0/342941787 learned_addr learned my addr 172.21.15.40:0/342941787 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6caffd640 1 --2- 172.21.15.40:0/342941787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6cc0adec0 0x7fd6cc0ac3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6cbfff640 1 --2- 172.21.15.40:0/342941787 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6cc159dd0 0x7fd6cc0aa1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:55.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6cb7fe640 1 -- 172.21.15.40:0/342941787 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6cc159dd0 msgr2=0x7fd6cc0aa1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6cb7fe640 1 --2- 172.21.15.40:0/342941787 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6cc159dd0 0x7fd6cc0aa1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6cb7fe640 1 -- 172.21.15.40:0/342941787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6cc0adec0 msgr2=0x7fd6cc0ac3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6cb7fe640 1 --2- 172.21.15.40:0/342941787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6cc0adec0 0x7fd6cc0ac3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6cb7fe640 1 -- 172.21.15.40:0/342941787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6c0009420 con 0x7fd6cc06b800 2023-10-16T08:38:55.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6caffd640 1 --2- 172.21.15.40:0/342941787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6cc0adec0 0x7fd6cc0ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6cbfff640 1 --2- 172.21.15.40:0/342941787 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6cc159dd0 0x7fd6cc0aa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6cb7fe640 1 --2- 172.21.15.40:0/342941787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6cc06b800 0x7fd6cc150180 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00c3c0 tx=0x7fd6bc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6b3fff640 1 -- 172.21.15.40:0/342941787 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc0052d0 con 0x7fd6cc06b800 2023-10-16T08:38:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6b3fff640 1 -- 172.21.15.40:0/342941787 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6bc005430 con 0x7fd6cc06b800 2023-10-16T08:38:55.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6d21c0640 1 -- 172.21.15.40:0/342941787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6cc1577e0 con 0x7fd6cc06b800 2023-10-16T08:38:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6b3fff640 1 -- 172.21.15.40:0/342941787 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6bc003840 con 0x7fd6cc06b800 2023-10-16T08:38:55.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.072+0000 7fd6d21c0640 1 -- 172.21.15.40:0/342941787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6cc0ac8e0 con 0x7fd6cc06b800 2023-10-16T08:38:55.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.076+0000 7fd6d21c0640 1 -- 172.21.15.40:0/342941787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6cc067a20 con 0x7fd6cc06b800 2023-10-16T08:38:55.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.076+0000 7fd6b3fff640 1 -- 172.21.15.40:0/342941787 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd6bc0039a0 con 0x7fd6cc06b800 2023-10-16T08:38:55.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.076+0000 7fd6b3fff640 1 --2- 172.21.15.40:0/342941787 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd6b80664a0 0x7fd6b8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:55.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.076+0000 7fd6b3fff640 1 -- 172.21.15.40:0/342941787 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd6bc083450 con 0x7fd6cc06b800 2023-10-16T08:38:55.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.076+0000 7fd6caffd640 1 --2- 172.21.15.40:0/342941787 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd6b80664a0 0x7fd6b8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:55.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.076+0000 7fd6caffd640 1 --2- 172.21.15.40:0/342941787 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd6b80664a0 0x7fd6b8068960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00ae000 tx=0x7fd6c0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:55.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.080+0000 7fd6b3fff640 1 -- 172.21.15.40:0/342941787 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fd6bc050830 con 0x7fd6cc06b800 2023-10-16T08:38:55.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.232+0000 7fd6d21c0640 1 -- 172.21.15.40:0/342941787 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd6cc0ce690 con 0x7fd6cc06b800 2023-10-16T08:38:55.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.232+0000 7fd6b3fff640 1 -- 172.21.15.40:0/342941787 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd6bc053e50 con 0x7fd6cc06b800 2023-10-16T08:38:55.236 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-16T08:38:55.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.232+0000 7fd6d21c0640 1 -- 172.21.15.40:0/342941787 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd6b80664a0 msgr2=0x7fd6b8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.232+0000 7fd6d21c0640 1 --2- 172.21.15.40:0/342941787 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd6b80664a0 0x7fd6b8068960 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fd6c00ae000 tx=0x7fd6c0008040 comp rx=0 tx=0).stop 2023-10-16T08:38:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.232+0000 7fd6d21c0640 1 -- 172.21.15.40:0/342941787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6cc06b800 msgr2=0x7fd6cc150180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.232+0000 7fd6d21c0640 1 --2- 172.21.15.40:0/342941787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6cc06b800 0x7fd6cc150180 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fd6bc00c3c0 tx=0x7fd6bc00c880 comp rx=0 tx=0).stop 2023-10-16T08:38:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.232+0000 7fd6d21c0640 1 -- 172.21.15.40:0/342941787 shutdown_connections 2023-10-16T08:38:55.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.232+0000 7fd6d21c0640 1 --2- 172.21.15.40:0/342941787 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd6cc159dd0 0x7fd6cc0aa1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.232+0000 7fd6d21c0640 1 --2- 172.21.15.40:0/342941787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6cc0adec0 0x7fd6cc0ac3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.232+0000 7fd6d21c0640 1 --2- 172.21.15.40:0/342941787 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd6b80664a0 0x7fd6b8068960 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.232+0000 7fd6d21c0640 1 --2- 172.21.15.40:0/342941787 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd6cc06b800 0x7fd6cc150180 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.232+0000 7fd6d21c0640 1 -- 172.21.15.40:0/342941787 >> 172.21.15.40:0/342941787 conn(0x7fd6cc0ce890 msgr2=0x7fd6cc152680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:55.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.232+0000 7fd6d21c0640 1 -- 172.21.15.40:0/342941787 shutdown_connections 2023-10-16T08:38:55.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.232+0000 7fd6d21c0640 1 -- 172.21.15.40:0/342941787 wait complete. 2023-10-16T08:38:55.254 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:38:55.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.372+0000 7f738a1eb640 1 Processor -- start 2023-10-16T08:38:55.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.372+0000 7f738a1eb640 1 -- start start 2023-10-16T08:38:55.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.372+0000 7f738a1eb640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f738415d180 0x7f738406b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:55.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.372+0000 7f738a1eb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f738415d560 0x7f738406fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:55.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.372+0000 7f738a1eb640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f73840ad820 0x7f73840c9570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:55.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.372+0000 7f738a1eb640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7384070090 con 0x7f73840ad820 2023-10-16T08:38:55.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.372+0000 7f738a1eb640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73840cf480 con 0x7f738415d180 2023-10-16T08:38:55.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.372+0000 7f738a1eb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73840cf760 con 0x7f738415d560 2023-10-16T08:38:55.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.372+0000 7f7383fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f73840ad820 0x7f73840c9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:55.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.372+0000 7f7383fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f73840ad820 0x7f73840c9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34032/0 (socket says 172.21.15.40:34032) 2023-10-16T08:38:55.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.372+0000 7f7383fff640 1 -- 172.21.15.40:0/3506005425 learned_addr learned my addr 172.21.15.40:0/3506005425 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:55.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.372+0000 7f7382ffd640 1 --2- 172.21.15.40:0/3506005425 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f738415d560 0x7f738406fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:55.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.372+0000 7f73837fe640 1 --2- 172.21.15.40:0/3506005425 >> v2:172.21.15.86:3300/0 conn(0x7f738415d180 0x7f738406b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:55.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.372+0000 7f7383fff640 1 -- 172.21.15.40:0/3506005425 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f738415d560 msgr2=0x7f738406fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:55.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.372+0000 7f7383fff640 1 --2- 172.21.15.40:0/3506005425 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f738415d560 0x7f738406fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.376+0000 7f7383fff640 1 -- 172.21.15.40:0/3506005425 >> v2:172.21.15.86:3300/0 conn(0x7f738415d180 msgr2=0x7f738406b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:55.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.376+0000 7f7383fff640 1 --2- 172.21.15.40:0/3506005425 >> v2:172.21.15.86:3300/0 conn(0x7f738415d180 0x7f738406b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.376+0000 7f7383fff640 1 -- 172.21.15.40:0/3506005425 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73840c8fb0 con 0x7f73840ad820 2023-10-16T08:38:55.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.376+0000 7f7383fff640 1 --2- 172.21.15.40:0/3506005425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73840ad820 0x7f73840c9570 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f738406a0d0 tx=0x7f737c01eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38df37caaa47091 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:55.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.376+0000 7f73827fc640 1 -- 172.21.15.40:0/3506005425 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c00b2d0 con 0x7f73840ad820 2023-10-16T08:38:55.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.376+0000 7f73827fc640 1 -- 172.21.15.40:0/3506005425 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c00b430 con 0x7f73840ad820 2023-10-16T08:38:55.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.376+0000 7f73827fc640 1 -- 172.21.15.40:0/3506005425 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c0a7c50 con 0x7f73840ad820 2023-10-16T08:38:55.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.376+0000 7f738a1eb640 1 -- 172.21.15.40:0/3506005425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73840ad820 msgr2=0x7f73840c9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:55.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.376+0000 7f738a1eb640 1 --2- 172.21.15.40:0/3506005425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73840ad820 0x7f73840c9570 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f738406a0d0 tx=0x7f737c01eb80 comp rx=0 tx=0).stop 2023-10-16T08:38:55.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.376+0000 7f738a1eb640 1 -- 172.21.15.40:0/3506005425 shutdown_connections 2023-10-16T08:38:55.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.376+0000 7f738a1eb640 1 --2- 172.21.15.40:0/3506005425 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73840ad820 0x7f73840c9570 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.376+0000 7f738a1eb640 1 --2- 172.21.15.40:0/3506005425 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f738415d560 0x7f738406fb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.376+0000 7f738a1eb640 1 --2- 172.21.15.40:0/3506005425 >> v2:172.21.15.86:3300/0 conn(0x7f738415d180 0x7f738406b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.376+0000 7f738a1eb640 1 -- 172.21.15.40:0/3506005425 >> 172.21.15.40:0/3506005425 conn(0x7f73840ce960 msgr2=0x7f73840ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:55.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.376+0000 7f738a1eb640 1 -- 172.21.15.40:0/3506005425 shutdown_connections 2023-10-16T08:38:55.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.376+0000 7f738a1eb640 1 -- 172.21.15.40:0/3506005425 wait complete. 2023-10-16T08:38:55.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f738a1eb640 1 Processor -- start 2023-10-16T08:38:55.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f738a1eb640 1 -- start start 2023-10-16T08:38:55.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f738a1eb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73840ad820 0x7f7384145b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:55.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f738a1eb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f738415d180 0x7f73840af040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:55.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f738a1eb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f738415d560 0x7f73840af580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:55.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f738a1eb640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73841594b0 con 0x7f738415d560 2023-10-16T08:38:55.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f738a1eb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f738415c330 con 0x7f73840ad820 2023-10-16T08:38:55.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f738a1eb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7384159020 con 0x7f738415d180 2023-10-16T08:38:55.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f7383fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f738415d560 0x7f73840af580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:55.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f7383fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f738415d560 0x7f73840af580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34042/0 (socket says 172.21.15.40:34042) 2023-10-16T08:38:55.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f7383fff640 1 -- 172.21.15.40:0/1090585476 learned_addr learned my addr 172.21.15.40:0/1090585476 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:55.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f73837fe640 1 --2- 172.21.15.40:0/1090585476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73840ad820 0x7f7384145b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:55.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f7382ffd640 1 --2- 172.21.15.40:0/1090585476 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f738415d180 0x7f73840af040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:55.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f7383fff640 1 -- 172.21.15.40:0/1090585476 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f738415d180 msgr2=0x7f73840af040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:55.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f7383fff640 1 --2- 172.21.15.40:0/1090585476 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f738415d180 0x7f73840af040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f7383fff640 1 -- 172.21.15.40:0/1090585476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73840ad820 msgr2=0x7f7384145b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:55.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f7383fff640 1 --2- 172.21.15.40:0/1090585476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73840ad820 0x7f7384145b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f7383fff640 1 -- 172.21.15.40:0/1090585476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f737c00bd40 con 0x7f738415d560 2023-10-16T08:38:55.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f7383fff640 1 --2- 172.21.15.40:0/1090585476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f738415d560 0x7f73840af580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f737c0b1040 tx=0x7f737c01b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:55.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f736bfff640 1 -- 172.21.15.40:0/1090585476 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c009800 con 0x7f738415d560 2023-10-16T08:38:55.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f736bfff640 1 -- 172.21.15.40:0/1090585476 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f737c009960 con 0x7f738415d560 2023-10-16T08:38:55.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f738a1eb640 1 -- 172.21.15.40:0/1090585476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73841460c0 con 0x7f738415d560 2023-10-16T08:38:55.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f736bfff640 1 -- 172.21.15.40:0/1090585476 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f737c009d10 con 0x7f738415d560 2023-10-16T08:38:55.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f738a1eb640 1 -- 172.21.15.40:0/1090585476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f738415c860 con 0x7f738415d560 2023-10-16T08:38:55.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f738a1eb640 1 -- 172.21.15.40:0/1090585476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73841591b0 con 0x7f738415d560 2023-10-16T08:38:55.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.380+0000 7f736bfff640 1 -- 172.21.15.40:0/1090585476 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f737c0ba470 con 0x7f738415d560 2023-10-16T08:38:55.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.384+0000 7f736bfff640 1 --2- 172.21.15.40:0/1090585476 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f73700664d0 0x7f7370068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:55.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.384+0000 7f736bfff640 1 -- 172.21.15.40:0/1090585476 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f737c121b90 con 0x7f738415d560 2023-10-16T08:38:55.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.384+0000 7f73837fe640 1 --2- 172.21.15.40:0/1090585476 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f73700664d0 0x7f7370068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:55.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.384+0000 7f73837fe640 1 --2- 172.21.15.40:0/1090585476 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f73700664d0 0x7f7370068990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f738415c0f0 tx=0x7f7374008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:55.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.384+0000 7f736bfff640 1 -- 172.21.15.40:0/1090585476 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f737c0ef2a0 con 0x7f738415d560 2023-10-16T08:38:55.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.572+0000 7f738a1eb640 1 -- 172.21.15.40:0/1090585476 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f738406d820 con 0x7f738415d560 2023-10-16T08:38:55.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.572+0000 7f736bfff640 1 -- 172.21.15.40:0/1090585476 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4898 (secure 0 0 0) 0x7f737c0be090 con 0x7f738415d560 2023-10-16T08:38:55.576 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:55.576 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/32529848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":32529848},{"type":"v1","addr":"172.21.15.86:6837","nonce":32529848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2044121591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2044121591},{"type":"v1","addr":"172.21.15.86:6836","nonce":2044121591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1275484997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1275484997},{"type":"v1","addr":"172.21.15.40:6833","nonce":1275484997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:38:44.984772+0000","modified":"2023-10-16T08:38:47.305255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/148507680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":148507680},{"type":"v1","addr":"172.21.15.40:6835","nonce":148507680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T08:38:55.577 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 6 2023-10-16T08:38:55.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.572+0000 7f738a1eb640 1 -- 172.21.15.40:0/1090585476 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f73700664d0 msgr2=0x7f7370068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:55.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.572+0000 7f738a1eb640 1 --2- 172.21.15.40:0/1090585476 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f73700664d0 0x7f7370068990 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f738415c0f0 tx=0x7f7374008040 comp rx=0 tx=0).stop 2023-10-16T08:38:55.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.572+0000 7f738a1eb640 1 -- 172.21.15.40:0/1090585476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f738415d560 msgr2=0x7f73840af580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:55.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.572+0000 7f738a1eb640 1 --2- 172.21.15.40:0/1090585476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f738415d560 0x7f73840af580 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f737c0b1040 tx=0x7f737c01b920 comp rx=0 tx=0).stop 2023-10-16T08:38:55.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.572+0000 7f738a1eb640 1 -- 172.21.15.40:0/1090585476 shutdown_connections 2023-10-16T08:38:55.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.572+0000 7f738a1eb640 1 --2- 172.21.15.40:0/1090585476 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f73700664d0 0x7f7370068990 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.572+0000 7f738a1eb640 1 --2- 172.21.15.40:0/1090585476 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f738415d560 0x7f73840af580 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.572+0000 7f738a1eb640 1 --2- 172.21.15.40:0/1090585476 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f738415d180 0x7f73840af040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.572+0000 7f738a1eb640 1 --2- 172.21.15.40:0/1090585476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f73840ad820 0x7f7384145b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.572+0000 7f738a1eb640 1 -- 172.21.15.40:0/1090585476 >> 172.21.15.40:0/1090585476 conn(0x7f73840ce960 msgr2=0x7f73840cbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.572+0000 7f738a1eb640 1 -- 172.21.15.40:0/1090585476 shutdown_connections 2023-10-16T08:38:55.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.572+0000 7f738a1eb640 1 -- 172.21.15.40:0/1090585476 wait complete. 2023-10-16T08:38:55.594 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:38:55.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.712+0000 7fd40e3b3640 1 Processor -- start 2023-10-16T08:38:55.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.712+0000 7fd40e3b3640 1 -- start start 2023-10-16T08:38:55.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.712+0000 7fd40e3b3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd40815d560 0x7fd40806b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:55.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.712+0000 7fd40e3b3640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4080cf480 con 0x7fd4080ad820 2023-10-16T08:38:55.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.712+0000 7fd40e3b3640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4080cf760 con 0x7fd40815d180 2023-10-16T08:38:55.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.712+0000 7fd40e3b3640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4080c8fb0 con 0x7fd40815d560 2023-10-16T08:38:55.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.712+0000 7fd3fffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd40815d560 0x7fd40806b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:55.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.712+0000 7fd40c950640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fd4080ad820 0x7fd4080cd220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50400/0 (socket says 172.21.15.40:50400) 2023-10-16T08:38:55.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.712+0000 7fd40c950640 1 -- 172.21.15.40:0/2719575378 learned_addr learned my addr 172.21.15.40:0/2719575378 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:55.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.712+0000 7fd3fffff640 1 -- 172.21.15.40:0/2719575378 >> v1:172.21.15.86:6789/0 conn(0x7fd40815d180 legacy=0x7fd4080c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:55.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.712+0000 7fd3fffff640 1 -- 172.21.15.40:0/2719575378 >> v1:172.21.15.40:6789/0 conn(0x7fd4080ad820 legacy=0x7fd4080cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:55.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.712+0000 7fd3fffff640 1 -- 172.21.15.40:0/2719575378 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4080c9290 con 0x7fd40815d560 2023-10-16T08:38:55.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd3fffff640 1 --2- 172.21.15.40:0/2719575378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd40815d560 0x7fd40806b7e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd3f000ba60 tx=0x7fd3f001eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=feaa992f69b76d97 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:55.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd4077fe640 1 -- 172.21.15.40:0/2719575378 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f00a7ac0 con 0x7fd40815d560 2023-10-16T08:38:55.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd4077fe640 1 -- 172.21.15.40:0/2719575378 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f00a7c20 con 0x7fd40815d560 2023-10-16T08:38:55.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd4077fe640 1 -- 172.21.15.40:0/2719575378 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f00af440 con 0x7fd40815d560 2023-10-16T08:38:55.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd40e3b3640 1 -- 172.21.15.40:0/2719575378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd40815d560 msgr2=0x7fd40806b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:55.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd40e3b3640 1 --2- 172.21.15.40:0/2719575378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd40815d560 0x7fd40806b7e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd3f000ba60 tx=0x7fd3f001eb10 comp rx=0 tx=0).stop 2023-10-16T08:38:55.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd40e3b3640 1 -- 172.21.15.40:0/2719575378 shutdown_connections 2023-10-16T08:38:55.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd40e3b3640 1 --2- 172.21.15.40:0/2719575378 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd40815d560 0x7fd40806b7e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd40e3b3640 1 -- 172.21.15.40:0/2719575378 >> 172.21.15.40:0/2719575378 conn(0x7fd4080ce960 msgr2=0x7fd4080ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:55.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd40e3b3640 1 -- 172.21.15.40:0/2719575378 shutdown_connections 2023-10-16T08:38:55.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd40e3b3640 1 -- 172.21.15.40:0/2719575378 wait complete. 2023-10-16T08:38:55.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd40e3b3640 1 Processor -- start 2023-10-16T08:38:55.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd40e3b3640 1 -- start start 2023-10-16T08:38:55.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd40e3b3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4080ad820 0x7fd4080700e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:55.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd40e3b3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd40815d180 0x7fd408070620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:55.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd40e3b3640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd40815d560 0x7fd40814fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:55.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd40e3b3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd40814e3c0 con 0x7fd4080ad820 2023-10-16T08:38:55.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd40e3b3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4080b1580 con 0x7fd40815d180 2023-10-16T08:38:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd40e3b3640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd40814d780 con 0x7fd40815d560 2023-10-16T08:38:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd407fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4080ad820 0x7fd4080700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd407fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4080ad820 0x7fd4080700e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34062/0 (socket says 172.21.15.40:34062) 2023-10-16T08:38:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd407fff640 1 -- 172.21.15.40:0/989030721 learned_addr learned my addr 172.21.15.40:0/989030721 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:55.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd407fff640 1 -- 172.21.15.40:0/989030721 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd40815d560 msgr2=0x7fd40814fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd407fff640 1 --2- 172.21.15.40:0/989030721 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd40815d560 0x7fd40814fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd407fff640 1 -- 172.21.15.40:0/989030721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd40815d180 msgr2=0x7fd408070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd407fff640 1 --2- 172.21.15.40:0/989030721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd40815d180 0x7fd408070620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd407fff640 1 -- 172.21.15.40:0/989030721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f000b6c0 con 0x7fd4080ad820 2023-10-16T08:38:55.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd407fff640 1 --2- 172.21.15.40:0/989030721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4080ad820 0x7fd4080700e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800c450 tx=0x7fd3f800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:55.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd4057fa640 1 -- 172.21.15.40:0/989030721 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8012830 con 0x7fd4080ad820 2023-10-16T08:38:55.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd4057fa640 1 -- 172.21.15.40:0/989030721 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f80129b0 con 0x7fd4080ad820 2023-10-16T08:38:55.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd40e3b3640 1 -- 172.21.15.40:0/989030721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd40814da20 con 0x7fd4080ad820 2023-10-16T08:38:55.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd4057fa640 1 -- 172.21.15.40:0/989030721 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f8012d60 con 0x7fd4080ad820 2023-10-16T08:38:55.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.716+0000 7fd40e3b3640 1 -- 172.21.15.40:0/989030721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd40814dfe0 con 0x7fd4080ad820 2023-10-16T08:38:55.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.720+0000 7fd40e3b3640 1 -- 172.21.15.40:0/989030721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd40814e630 con 0x7fd4080ad820 2023-10-16T08:38:55.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.720+0000 7fd4057fa640 1 -- 172.21.15.40:0/989030721 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd3f801c460 con 0x7fd4080ad820 2023-10-16T08:38:55.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.720+0000 7fd4057fa640 1 --2- 172.21.15.40:0/989030721 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd3e80664a0 0x7fd3e8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:55.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.720+0000 7fd4057fa640 1 -- 172.21.15.40:0/989030721 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd3f8083c40 con 0x7fd4080ad820 2023-10-16T08:38:55.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.720+0000 7fd3fffff640 1 --2- 172.21.15.40:0/989030721 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd3e80664a0 0x7fd3e8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:55.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.720+0000 7fd3fffff640 1 --2- 172.21.15.40:0/989030721 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd3e80664a0 0x7fd3e8068960 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd3f00b1000 tx=0x7fd3f0004740 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:55.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.724+0000 7fd4057fa640 1 -- 172.21.15.40:0/989030721 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fd3f8051150 con 0x7fd4080ad820 2023-10-16T08:38:55.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.876+0000 7fd40e3b3640 1 -- 172.21.15.40:0/989030721 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd40804e9c0 con 0x7fd4080ad820 2023-10-16T08:38:55.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.876+0000 7fd4057fa640 1 -- 172.21.15.40:0/989030721 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16445 (secure 0 0 0) 0x7fd3f8054770 con 0x7fd4080ad820 2023-10-16T08:38:55.883 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:55.883 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:46.611098+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:38:55.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.880+0000 7fd40e3b3640 1 -- 172.21.15.40:0/989030721 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd3e80664a0 msgr2=0x7fd3e8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:55.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.880+0000 7fd40e3b3640 1 --2- 172.21.15.40:0/989030721 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd3e80664a0 0x7fd3e8068960 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fd3f00b1000 tx=0x7fd3f0004740 comp rx=0 tx=0).stop 2023-10-16T08:38:55.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.880+0000 7fd40e3b3640 1 -- 172.21.15.40:0/989030721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4080ad820 msgr2=0x7fd4080700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:55.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.880+0000 7fd40e3b3640 1 --2- 172.21.15.40:0/989030721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4080ad820 0x7fd4080700e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd3f800c450 tx=0x7fd3f800c910 comp rx=0 tx=0).stop 2023-10-16T08:38:55.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.880+0000 7fd40e3b3640 1 -- 172.21.15.40:0/989030721 shutdown_connections 2023-10-16T08:38:55.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.880+0000 7fd40e3b3640 1 --2- 172.21.15.40:0/989030721 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd40815d560 0x7fd40814fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.880+0000 7fd40e3b3640 1 --2- 172.21.15.40:0/989030721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd40815d180 0x7fd408070620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.880+0000 7fd40e3b3640 1 --2- 172.21.15.40:0/989030721 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd3e80664a0 0x7fd3e8068960 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.880+0000 7fd40e3b3640 1 --2- 172.21.15.40:0/989030721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd4080ad820 0x7fd4080700e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:55.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.880+0000 7fd40e3b3640 1 -- 172.21.15.40:0/989030721 >> 172.21.15.40:0/989030721 conn(0x7fd4080ce960 msgr2=0x7fd4080ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:55.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.880+0000 7fd40e3b3640 1 -- 172.21.15.40:0/989030721 shutdown_connections 2023-10-16T08:38:55.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:55.880+0000 7fd40e3b3640 1 -- 172.21.15.40:0/989030721 wait complete. 2023-10-16T08:38:55.900 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:38:33.014668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:38:43.055215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:38:43.971666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:38:55.901 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:38:56.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.024+0000 7fc95f807640 1 Processor -- start 2023-10-16T08:38:56.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.024+0000 7fc95f807640 1 -- start start 2023-10-16T08:38:56.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.028+0000 7fc95f807640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc958159dd0 0x7fc9580c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.028+0000 7fc95f807640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc9580c9f00 0x7fc9580cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.028+0000 7fc95f807640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc95815b490 con 0x7fc9580adec0 2023-10-16T08:38:56.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.028+0000 7fc95f807640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc95806cd70 con 0x7fc958159dd0 2023-10-16T08:38:56.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.028+0000 7fc95f807640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95806d230 con 0x7fc9580c9f00 2023-10-16T08:38:56.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.028+0000 7fc95d5a3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc958159dd0 0x7fc9580c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.028+0000 7fc95cda2640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc9580adec0 0x7fc9580c97f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50406/0 (socket says 172.21.15.40:50406) 2023-10-16T08:38:56.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.028+0000 7fc95cda2640 1 -- 172.21.15.40:0/230593721 learned_addr learned my addr 172.21.15.40:0/230593721 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:56.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.028+0000 7fc95d5a3640 1 -- 172.21.15.40:0/230593721 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc9580c9f00 msgr2=0x7fc9580cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:56.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.028+0000 7fc95dda4640 1 --2- 172.21.15.40:0/230593721 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc9580c9f00 0x7fc9580cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.028+0000 7fc95d5a3640 1 --2- 172.21.15.40:0/230593721 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc9580c9f00 0x7fc9580cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.028+0000 7fc95d5a3640 1 -- 172.21.15.40:0/230593721 >> v1:172.21.15.40:6789/0 conn(0x7fc9580adec0 legacy=0x7fc9580c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.028+0000 7fc95d5a3640 1 -- 172.21.15.40:0/230593721 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc95806cf50 con 0x7fc958159dd0 2023-10-16T08:38:56.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.028+0000 7fc94ffff640 1 -- 172.21.15.40:0/230593721 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4199043113 0 0) 0x7fc95815b490 con 0x7fc9580adec0 2023-10-16T08:38:56.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.028+0000 7fc95dda4640 1 --2- 172.21.15.40:0/230593721 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc9580c9f00 0x7fc9580cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:56.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.028+0000 7fc95d5a3640 1 --2- 172.21.15.40:0/230593721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc958159dd0 0x7fc9580c8f70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc9540097c0 tx=0x7fc95401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2124ec7779ade4c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:56.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.028+0000 7fc94ffff640 1 -- 172.21.15.40:0/230593721 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9540ab600 con 0x7fc958159dd0 2023-10-16T08:38:56.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.028+0000 7fc94ffff640 1 -- 172.21.15.40:0/230593721 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9540a7050 con 0x7fc958159dd0 2023-10-16T08:38:56.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.028+0000 7fc94ffff640 1 -- 172.21.15.40:0/230593721 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9540ab980 con 0x7fc958159dd0 2023-10-16T08:38:56.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95f807640 1 -- 172.21.15.40:0/230593721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc958159dd0 msgr2=0x7fc9580c8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95f807640 1 --2- 172.21.15.40:0/230593721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc958159dd0 0x7fc9580c8f70 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc9540097c0 tx=0x7fc95401c480 comp rx=0 tx=0).stop 2023-10-16T08:38:56.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95f807640 1 -- 172.21.15.40:0/230593721 shutdown_connections 2023-10-16T08:38:56.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95f807640 1 --2- 172.21.15.40:0/230593721 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc9580c9f00 0x7fc9580cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95f807640 1 --2- 172.21.15.40:0/230593721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc958159dd0 0x7fc9580c8f70 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95f807640 1 -- 172.21.15.40:0/230593721 >> 172.21.15.40:0/230593721 conn(0x7fc9580ce890 msgr2=0x7fc9580ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:56.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95f807640 1 -- 172.21.15.40:0/230593721 shutdown_connections 2023-10-16T08:38:56.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95f807640 1 -- 172.21.15.40:0/230593721 wait complete. 2023-10-16T08:38:56.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95f807640 1 Processor -- start 2023-10-16T08:38:56.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95f807640 1 -- start start 2023-10-16T08:38:56.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95f807640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9580adec0 0x7fc95815d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95f807640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9580c9f00 0x7fc95815bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95f807640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc958159dd0 0x7fc9580acb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95f807640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95815da30 con 0x7fc9580c9f00 2023-10-16T08:38:56.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95f807640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc95815c450 con 0x7fc9580adec0 2023-10-16T08:38:56.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95f807640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc95815c730 con 0x7fc958159dd0 2023-10-16T08:38:56.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95cda2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9580c9f00 0x7fc95815bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95cda2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9580c9f00 0x7fc95815bb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34078/0 (socket says 172.21.15.40:34078) 2023-10-16T08:38:56.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95cda2640 1 -- 172.21.15.40:0/2355255436 learned_addr learned my addr 172.21.15.40:0/2355255436 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:56.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95d5a3640 1 --2- 172.21.15.40:0/2355255436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9580adec0 0x7fc95815d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95dda4640 1 --2- 172.21.15.40:0/2355255436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc958159dd0 0x7fc9580acb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95cda2640 1 -- 172.21.15.40:0/2355255436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc958159dd0 msgr2=0x7fc9580acb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95cda2640 1 --2- 172.21.15.40:0/2355255436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc958159dd0 0x7fc9580acb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95cda2640 1 -- 172.21.15.40:0/2355255436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9580adec0 msgr2=0x7fc95815d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95cda2640 1 --2- 172.21.15.40:0/2355255436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9580adec0 0x7fc95815d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95cda2640 1 -- 172.21.15.40:0/2355255436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc954009420 con 0x7fc9580c9f00 2023-10-16T08:38:56.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95d5a3640 1 --2- 172.21.15.40:0/2355255436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9580adec0 0x7fc95815d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:38:56.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95cda2640 1 --2- 172.21.15.40:0/2355255436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9580c9f00 0x7fc95815bb70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc94800c4b0 tx=0x7fc94800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:56.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc94dffb640 1 -- 172.21.15.40:0/2355255436 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9480074e0 con 0x7fc9580c9f00 2023-10-16T08:38:56.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc94dffb640 1 -- 172.21.15.40:0/2355255436 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc948007660 con 0x7fc9580c9f00 2023-10-16T08:38:56.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc94dffb640 1 -- 172.21.15.40:0/2355255436 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc948004a30 con 0x7fc9580c9f00 2023-10-16T08:38:56.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95f807640 1 -- 172.21.15.40:0/2355255436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc95815ca10 con 0x7fc9580c9f00 2023-10-16T08:38:56.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.032+0000 7fc95f807640 1 -- 172.21.15.40:0/2355255436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc95815cfd0 con 0x7fc9580c9f00 2023-10-16T08:38:56.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.036+0000 7fc95f807640 1 -- 172.21.15.40:0/2355255436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc958067a20 con 0x7fc9580c9f00 2023-10-16T08:38:56.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.036+0000 7fc94dffb640 1 -- 172.21.15.40:0/2355255436 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc948004b90 con 0x7fc9580c9f00 2023-10-16T08:38:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.036+0000 7fc94dffb640 1 --2- 172.21.15.40:0/2355255436 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fc93c0664a0 0x7fc93c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.036+0000 7fc94dffb640 1 -- 172.21.15.40:0/2355255436 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc948083060 con 0x7fc9580c9f00 2023-10-16T08:38:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.036+0000 7fc95d5a3640 1 --2- 172.21.15.40:0/2355255436 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fc93c0664a0 0x7fc93c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.036+0000 7fc95d5a3640 1 --2- 172.21.15.40:0/2355255436 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fc93c0664a0 0x7fc93c068960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc9540ad000 tx=0x7fc9540a6000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:56.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.040+0000 7fc94dffb640 1 -- 172.21.15.40:0/2355255436 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fc948050650 con 0x7fc9580c9f00 2023-10-16T08:38:56.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.196+0000 7fc95f807640 1 -- 172.21.15.40:0/2355255436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc95815d1f0 con 0x7fc9580c9f00 2023-10-16T08:38:56.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.196+0000 7fc94dffb640 1 -- 172.21.15.40:0/2355255436 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc948018020 con 0x7fc9580c9f00 2023-10-16T08:38:56.202 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-16T08:38:56.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.200+0000 7fc95f807640 1 -- 172.21.15.40:0/2355255436 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fc93c0664a0 msgr2=0x7fc93c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.200+0000 7fc95f807640 1 --2- 172.21.15.40:0/2355255436 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fc93c0664a0 0x7fc93c068960 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc9540ad000 tx=0x7fc9540a6000 comp rx=0 tx=0).stop 2023-10-16T08:38:56.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.200+0000 7fc95f807640 1 -- 172.21.15.40:0/2355255436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9580c9f00 msgr2=0x7fc95815bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.200+0000 7fc95f807640 1 --2- 172.21.15.40:0/2355255436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9580c9f00 0x7fc95815bb70 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc94800c4b0 tx=0x7fc94800c970 comp rx=0 tx=0).stop 2023-10-16T08:38:56.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.200+0000 7fc95f807640 1 -- 172.21.15.40:0/2355255436 shutdown_connections 2023-10-16T08:38:56.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.200+0000 7fc95f807640 1 --2- 172.21.15.40:0/2355255436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc958159dd0 0x7fc9580acb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.200+0000 7fc95f807640 1 --2- 172.21.15.40:0/2355255436 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fc93c0664a0 0x7fc93c068960 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.200+0000 7fc95f807640 1 --2- 172.21.15.40:0/2355255436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc9580c9f00 0x7fc95815bb70 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.200+0000 7fc95f807640 1 --2- 172.21.15.40:0/2355255436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9580adec0 0x7fc95815d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.200+0000 7fc95f807640 1 -- 172.21.15.40:0/2355255436 >> 172.21.15.40:0/2355255436 conn(0x7fc9580ce890 msgr2=0x7fc958156200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:56.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.200+0000 7fc95f807640 1 -- 172.21.15.40:0/2355255436 shutdown_connections 2023-10-16T08:38:56.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.200+0000 7fc95f807640 1 -- 172.21.15.40:0/2355255436 wait complete. 2023-10-16T08:38:56.221 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:38:56.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.344+0000 7f7377239640 1 Processor -- start 2023-10-16T08:38:56.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.344+0000 7f7377239640 1 -- start start 2023-10-16T08:38:56.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f7377239640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f737015abb0 0x7f737015af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f7377239640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f73700afe60 0x7f73700b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f7377239640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f73700cc6e0 con 0x7f737015b4d0 2023-10-16T08:38:56.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f7377239640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73700cc9c0 con 0x7f737015abb0 2023-10-16T08:38:56.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f7377239640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73700ccca0 con 0x7f73700afe60 2023-10-16T08:38:56.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f7374fd5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f737015abb0 0x7f737015af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f7374fd5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f737015abb0 0x7f737015af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34086/0 (socket says 172.21.15.40:34086) 2023-10-16T08:38:56.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f7374fd5640 1 -- 172.21.15.40:0/1664560560 learned_addr learned my addr 172.21.15.40:0/1664560560 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:56.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f736ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f737015b4d0 0x7f73700af750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50420/0 (socket says 172.21.15.40:50420) 2023-10-16T08:38:56.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f73757d6640 1 --2- 172.21.15.40:0/1664560560 >> v2:172.21.15.86:3300/0 conn(0x7f73700afe60 0x7f73700b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f7374fd5640 1 -- 172.21.15.40:0/1664560560 >> v2:172.21.15.86:3300/0 conn(0x7f73700afe60 msgr2=0x7f73700b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:56.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f7374fd5640 1 --2- 172.21.15.40:0/1664560560 >> v2:172.21.15.86:3300/0 conn(0x7f73700afe60 0x7f73700b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f7374fd5640 1 -- 172.21.15.40:0/1664560560 >> v1:172.21.15.40:6789/0 conn(0x7f737015b4d0 legacy=0x7f73700af750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f736f7fe640 1 -- 172.21.15.40:0/1664560560 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 377157574 0 0) 0x7f73700cc6e0 con 0x7f737015b4d0 2023-10-16T08:38:56.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f7374fd5640 1 -- 172.21.15.40:0/1664560560 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73700ccf80 con 0x7f737015abb0 2023-10-16T08:38:56.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f7374fd5640 1 --2- 172.21.15.40:0/1664560560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f737015abb0 0x7f737015af90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f73700c79c0 tx=0x7f736801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0f87f7a1113ae1b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:56.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f736f7fe640 1 -- 172.21.15.40:0/1664560560 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73680a8050 con 0x7f737015abb0 2023-10-16T08:38:56.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f736f7fe640 1 -- 172.21.15.40:0/1664560560 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73680ac5f0 con 0x7f737015abb0 2023-10-16T08:38:56.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f736f7fe640 1 -- 172.21.15.40:0/1664560560 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73680ac980 con 0x7f737015abb0 2023-10-16T08:38:56.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f7377239640 1 -- 172.21.15.40:0/1664560560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f737015abb0 msgr2=0x7f737015af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f7377239640 1 --2- 172.21.15.40:0/1664560560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f737015abb0 0x7f737015af90 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f73700c79c0 tx=0x7f736801c500 comp rx=0 tx=0).stop 2023-10-16T08:38:56.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f7377239640 1 -- 172.21.15.40:0/1664560560 shutdown_connections 2023-10-16T08:38:56.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f7377239640 1 --2- 172.21.15.40:0/1664560560 >> v2:172.21.15.86:3300/0 conn(0x7f73700afe60 0x7f73700b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f7377239640 1 --2- 172.21.15.40:0/1664560560 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f737015abb0 0x7f737015af90 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.348+0000 7f7377239640 1 -- 172.21.15.40:0/1664560560 >> 172.21.15.40:0/1664560560 conn(0x7f73700cd3c0 msgr2=0x7f73700cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:56.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7377239640 1 -- 172.21.15.40:0/1664560560 shutdown_connections 2023-10-16T08:38:56.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7377239640 1 -- 172.21.15.40:0/1664560560 wait complete. 2023-10-16T08:38:56.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7377239640 1 Processor -- start 2023-10-16T08:38:56.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7377239640 1 -- start start 2023-10-16T08:38:56.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7377239640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73700afe60 0x7f73700b17d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7377239640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f737015abb0 0x7f7370155a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7377239640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f737015b4d0 0x7f7370147ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7377239640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7370149660 con 0x7f73700afe60 2023-10-16T08:38:56.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7377239640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7370149940 con 0x7f737015b4d0 2023-10-16T08:38:56.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7377239640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73700c8d50 con 0x7f737015abb0 2023-10-16T08:38:56.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7374fd5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73700afe60 0x7f73700b17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7374fd5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73700afe60 0x7f73700b17d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34094/0 (socket says 172.21.15.40:34094) 2023-10-16T08:38:56.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7374fd5640 1 -- 172.21.15.40:0/2019249660 learned_addr learned my addr 172.21.15.40:0/2019249660 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:56.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f736ffff640 1 --2- 172.21.15.40:0/2019249660 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f737015abb0 0x7f7370155a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f73757d6640 1 --2- 172.21.15.40:0/2019249660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f737015b4d0 0x7f7370147ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7374fd5640 1 -- 172.21.15.40:0/2019249660 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f737015abb0 msgr2=0x7f7370155a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7374fd5640 1 --2- 172.21.15.40:0/2019249660 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f737015abb0 0x7f7370155a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7374fd5640 1 -- 172.21.15.40:0/2019249660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f737015b4d0 msgr2=0x7f7370147ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7374fd5640 1 --2- 172.21.15.40:0/2019249660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f737015b4d0 0x7f7370147ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7374fd5640 1 -- 172.21.15.40:0/2019249660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73680094d0 con 0x7f73700afe60 2023-10-16T08:38:56.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7374fd5640 1 --2- 172.21.15.40:0/2019249660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73700afe60 0x7f73700b17d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f73680af040 tx=0x7f73680192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:56.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f736d7fa640 1 -- 172.21.15.40:0/2019249660 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73680a8040 con 0x7f73700afe60 2023-10-16T08:38:56.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f736d7fa640 1 -- 172.21.15.40:0/2019249660 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73680ab820 con 0x7f73700afe60 2023-10-16T08:38:56.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7377239640 1 -- 172.21.15.40:0/2019249660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73700c9030 con 0x7f73700afe60 2023-10-16T08:38:56.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f736d7fa640 1 -- 172.21.15.40:0/2019249660 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73680abbb0 con 0x7f73700afe60 2023-10-16T08:38:56.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7377239640 1 -- 172.21.15.40:0/2019249660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73700c89b0 con 0x7f73700afe60 2023-10-16T08:38:56.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.352+0000 7f7377239640 1 -- 172.21.15.40:0/2019249660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73700c8ba0 con 0x7f73700afe60 2023-10-16T08:38:56.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.356+0000 7f736d7fa640 1 -- 172.21.15.40:0/2019249660 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f73680abd10 con 0x7f73700afe60 2023-10-16T08:38:56.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.356+0000 7f736d7fa640 1 --2- 172.21.15.40:0/2019249660 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f73500664d0 0x7f7350068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.356+0000 7f736d7fa640 1 -- 172.21.15.40:0/2019249660 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f736811e3e0 con 0x7f73700afe60 2023-10-16T08:38:56.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.356+0000 7f736ffff640 1 --2- 172.21.15.40:0/2019249660 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f73500664d0 0x7f7350068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.356+0000 7f736ffff640 1 --2- 172.21.15.40:0/2019249660 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f73500664d0 0x7f7350068990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7358007fd0 tx=0x7f7358009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:56.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.360+0000 7f736d7fa640 1 -- 172.21.15.40:0/2019249660 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f73680eb860 con 0x7f73700afe60 2023-10-16T08:38:56.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.516+0000 7f7377239640 1 -- 172.21.15.40:0/2019249660 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f7370149ab0 con 0x7f73700afe60 2023-10-16T08:38:56.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.516+0000 7f736d7fa640 1 -- 172.21.15.40:0/2019249660 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f73680eee80 con 0x7f73700afe60 2023-10-16T08:38:56.521 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-16T08:38:56.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.516+0000 7f7377239640 1 -- 172.21.15.40:0/2019249660 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f73500664d0 msgr2=0x7f7350068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.516+0000 7f7377239640 1 --2- 172.21.15.40:0/2019249660 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f73500664d0 0x7f7350068990 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f7358007fd0 tx=0x7f7358009040 comp rx=0 tx=0).stop 2023-10-16T08:38:56.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.516+0000 7f7377239640 1 -- 172.21.15.40:0/2019249660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73700afe60 msgr2=0x7f73700b17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.516+0000 7f7377239640 1 --2- 172.21.15.40:0/2019249660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73700afe60 0x7f73700b17d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f73680af040 tx=0x7f73680192a0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.516+0000 7f7377239640 1 -- 172.21.15.40:0/2019249660 shutdown_connections 2023-10-16T08:38:56.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.516+0000 7f7377239640 1 --2- 172.21.15.40:0/2019249660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f737015b4d0 0x7f7370147ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.516+0000 7f7377239640 1 --2- 172.21.15.40:0/2019249660 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f737015abb0 0x7f7370155a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.516+0000 7f7377239640 1 --2- 172.21.15.40:0/2019249660 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f73500664d0 0x7f7350068990 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.516+0000 7f7377239640 1 --2- 172.21.15.40:0/2019249660 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f73700afe60 0x7f73700b17d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.520+0000 7f7377239640 1 -- 172.21.15.40:0/2019249660 >> 172.21.15.40:0/2019249660 conn(0x7f73700cd3c0 msgr2=0x7f73700cd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:56.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.520+0000 7f7377239640 1 -- 172.21.15.40:0/2019249660 shutdown_connections 2023-10-16T08:38:56.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.520+0000 7f7377239640 1 -- 172.21.15.40:0/2019249660 wait complete. 2023-10-16T08:38:56.539 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:38:56.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.660+0000 7f27d16b4640 1 Processor -- start 2023-10-16T08:38:56.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.660+0000 7f27d16b4640 1 -- start start 2023-10-16T08:38:56.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.660+0000 7f27d16b4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f27cc0adeb0 0x7f27cc0b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.660+0000 7f27d16b4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27cc0c98b0 con 0x7f27cc0cae10 2023-10-16T08:38:56.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.660+0000 7f27d16b4640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27cc0b47c0 con 0x7f27cc0adad0 2023-10-16T08:38:56.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.660+0000 7f27d16b4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27cc0c8f80 con 0x7f27cc0adeb0 2023-10-16T08:38:56.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.660+0000 7f27cb7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f27cc0adeb0 0x7f27cc0b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.660+0000 7f27ca7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f27cc0cae10 0x7f27cc15a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50422/0 (socket says 172.21.15.40:50422) 2023-10-16T08:38:56.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.660+0000 7f27ca7fc640 1 -- 172.21.15.40:0/3696352061 learned_addr learned my addr 172.21.15.40:0/3696352061 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:56.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.660+0000 7f27cb7fe640 1 -- 172.21.15.40:0/3696352061 >> v1:172.21.15.86:6789/0 conn(0x7f27cc0adad0 legacy=0x7f27cc0ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:56.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.660+0000 7f27c9ffb640 1 -- 172.21.15.40:0/3696352061 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2081182058 0 0) 0x7f27cc0c98b0 con 0x7f27cc0cae10 2023-10-16T08:38:56.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.660+0000 7f27cb7fe640 1 -- 172.21.15.40:0/3696352061 >> v1:172.21.15.40:6789/0 conn(0x7f27cc0cae10 legacy=0x7f27cc15a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.660+0000 7f27cb7fe640 1 -- 172.21.15.40:0/3696352061 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27cc0c9260 con 0x7f27cc0adeb0 2023-10-16T08:38:56.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27cb7fe640 1 --2- 172.21.15.40:0/3696352061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27cc0adeb0 0x7f27cc0b41e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f27c40097c0 tx=0x7f27c401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8710585fe7f501d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:56.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27c9ffb640 1 -- 172.21.15.40:0/3696352061 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c40a7070 con 0x7f27cc0adeb0 2023-10-16T08:38:56.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27c9ffb640 1 -- 172.21.15.40:0/3696352061 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27c40ab430 con 0x7f27cc0adeb0 2023-10-16T08:38:56.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27c9ffb640 1 -- 172.21.15.40:0/3696352061 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27c40ab7c0 con 0x7f27cc0adeb0 2023-10-16T08:38:56.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27d16b4640 1 -- 172.21.15.40:0/3696352061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27cc0adeb0 msgr2=0x7f27cc0b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27d16b4640 1 --2- 172.21.15.40:0/3696352061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27cc0adeb0 0x7f27cc0b41e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f27c40097c0 tx=0x7f27c401c410 comp rx=0 tx=0).stop 2023-10-16T08:38:56.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27d16b4640 1 -- 172.21.15.40:0/3696352061 shutdown_connections 2023-10-16T08:38:56.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27d16b4640 1 --2- 172.21.15.40:0/3696352061 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27cc0adeb0 0x7f27cc0b41e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27d16b4640 1 -- 172.21.15.40:0/3696352061 >> 172.21.15.40:0/3696352061 conn(0x7f27cc0ce890 msgr2=0x7f27cc0ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:56.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27d16b4640 1 -- 172.21.15.40:0/3696352061 shutdown_connections 2023-10-16T08:38:56.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27d16b4640 1 -- 172.21.15.40:0/3696352061 wait complete. 2023-10-16T08:38:56.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27d16b4640 1 Processor -- start 2023-10-16T08:38:56.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27d16b4640 1 -- start start 2023-10-16T08:38:56.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27d16b4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27cc0adad0 0x7f27cc1397f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27d16b4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27cc0adeb0 0x7f27cc159ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27d16b4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27cc0cae10 0x7f27cc15a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27d16b4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc139d30 con 0x7f27cc0adad0 2023-10-16T08:38:56.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27d16b4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc15be20 con 0x7f27cc0adeb0 2023-10-16T08:38:56.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27d16b4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27cc15c0c0 con 0x7f27cc0cae10 2023-10-16T08:38:56.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27caffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27cc0adad0 0x7f27cc1397f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27caffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27cc0adad0 0x7f27cc1397f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34108/0 (socket says 172.21.15.40:34108) 2023-10-16T08:38:56.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.664+0000 7f27caffd640 1 -- 172.21.15.40:0/4242909770 learned_addr learned my addr 172.21.15.40:0/4242909770 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:56.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.668+0000 7f27ca7fc640 1 --2- 172.21.15.40:0/4242909770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27cc0adeb0 0x7f27cc159ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.668+0000 7f27caffd640 1 -- 172.21.15.40:0/4242909770 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27cc0cae10 msgr2=0x7f27cc15a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.668+0000 7f27caffd640 1 --2- 172.21.15.40:0/4242909770 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27cc0cae10 0x7f27cc15a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.668+0000 7f27caffd640 1 -- 172.21.15.40:0/4242909770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27cc0adeb0 msgr2=0x7f27cc159ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.668+0000 7f27caffd640 1 --2- 172.21.15.40:0/4242909770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27cc0adeb0 0x7f27cc159ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.668+0000 7f27caffd640 1 -- 172.21.15.40:0/4242909770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27c4009420 con 0x7f27cc0adad0 2023-10-16T08:38:56.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.668+0000 7f27caffd640 1 --2- 172.21.15.40:0/4242909770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27cc0adad0 0x7f27cc1397f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00c7c0 tx=0x7f27bc00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:56.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.668+0000 7f27b37fe640 1 -- 172.21.15.40:0/4242909770 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc00aa50 con 0x7f27cc0adad0 2023-10-16T08:38:56.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.668+0000 7f27b37fe640 1 -- 172.21.15.40:0/4242909770 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27bc00abb0 con 0x7f27cc0adad0 2023-10-16T08:38:56.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.668+0000 7f27b37fe640 1 -- 172.21.15.40:0/4242909770 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27bc012420 con 0x7f27cc0adad0 2023-10-16T08:38:56.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.668+0000 7f27d16b4640 1 -- 172.21.15.40:0/4242909770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27cc15c360 con 0x7f27cc0adad0 2023-10-16T08:38:56.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.668+0000 7f27d16b4640 1 -- 172.21.15.40:0/4242909770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27cc15c920 con 0x7f27cc0adad0 2023-10-16T08:38:56.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.668+0000 7f27d16b4640 1 -- 172.21.15.40:0/4242909770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27cc067a20 con 0x7f27cc0adad0 2023-10-16T08:38:56.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.668+0000 7f27b37fe640 1 -- 172.21.15.40:0/4242909770 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f27bc012580 con 0x7f27cc0adad0 2023-10-16T08:38:56.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.668+0000 7f27b37fe640 1 --2- 172.21.15.40:0/4242909770 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f27b80664d0 0x7f27b8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.668+0000 7f27b37fe640 1 -- 172.21.15.40:0/4242909770 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f27bc084130 con 0x7f27cc0adad0 2023-10-16T08:38:56.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.668+0000 7f27ca7fc640 1 --2- 172.21.15.40:0/4242909770 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f27b80664d0 0x7f27b8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.668+0000 7f27ca7fc640 1 --2- 172.21.15.40:0/4242909770 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f27b80664d0 0x7f27b8068990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0ca4f0 tx=0x7f27c0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:56.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.672+0000 7f27b37fe640 1 -- 172.21.15.40:0/4242909770 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f27bc051520 con 0x7f27cc0adad0 2023-10-16T08:38:56.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.827+0000 7f27d16b4640 1 -- 172.21.15.40:0/4242909770 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f27cc04e9c0 con 0x7f27cc0adad0 2023-10-16T08:38:56.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.831+0000 7f27b37fe640 1 -- 172.21.15.40:0/4242909770 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f27bc054b40 con 0x7f27cc0adad0 2023-10-16T08:38:56.835 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-16T08:38:56.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.831+0000 7f27d16b4640 1 -- 172.21.15.40:0/4242909770 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f27b80664d0 msgr2=0x7f27b8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.831+0000 7f27d16b4640 1 --2- 172.21.15.40:0/4242909770 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f27b80664d0 0x7f27b8068990 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f27cc0ca4f0 tx=0x7f27c0009040 comp rx=0 tx=0).stop 2023-10-16T08:38:56.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.831+0000 7f27d16b4640 1 -- 172.21.15.40:0/4242909770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27cc0adad0 msgr2=0x7f27cc1397f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.831+0000 7f27d16b4640 1 --2- 172.21.15.40:0/4242909770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27cc0adad0 0x7f27cc1397f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f27bc00c7c0 tx=0x7f27bc00cc80 comp rx=0 tx=0).stop 2023-10-16T08:38:56.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.831+0000 7f27d16b4640 1 -- 172.21.15.40:0/4242909770 shutdown_connections 2023-10-16T08:38:56.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.831+0000 7f27d16b4640 1 --2- 172.21.15.40:0/4242909770 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f27cc0cae10 0x7f27cc15a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.831+0000 7f27d16b4640 1 --2- 172.21.15.40:0/4242909770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f27cc0adeb0 0x7f27cc159ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.831+0000 7f27d16b4640 1 --2- 172.21.15.40:0/4242909770 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f27b80664d0 0x7f27b8068990 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.831+0000 7f27d16b4640 1 --2- 172.21.15.40:0/4242909770 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f27cc0adad0 0x7f27cc1397f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.831+0000 7f27d16b4640 1 -- 172.21.15.40:0/4242909770 >> 172.21.15.40:0/4242909770 conn(0x7f27cc0ce890 msgr2=0x7f27cc0c5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:56.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.831+0000 7f27d16b4640 1 -- 172.21.15.40:0/4242909770 shutdown_connections 2023-10-16T08:38:56.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.831+0000 7f27d16b4640 1 -- 172.21.15.40:0/4242909770 wait complete. 2023-10-16T08:38:56.855 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:38:56.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.967+0000 7fb3e18c0640 1 Processor -- start 2023-10-16T08:38:56.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.967+0000 7fb3e18c0640 1 -- start start 2023-10-16T08:38:56.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.971+0000 7fb3e18c0640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb3dc15b040 0x7fb3dc0ca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.971+0000 7fb3e18c0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3dc06c7c0 0x7fb3dc0cac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.971+0000 7fb3e18c0640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3dc0adca0 con 0x7fb3dc0adec0 2023-10-16T08:38:56.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.971+0000 7fb3e18c0640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3dc0c5840 con 0x7fb3dc15b040 2023-10-16T08:38:56.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.971+0000 7fb3e18c0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc0cf890 con 0x7fb3dc06c7c0 2023-10-16T08:38:56.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.971+0000 7fb3daffd640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb3dc15b040 0x7fb3dc0ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.971+0000 7fb3db7fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3dc06c7c0 0x7fb3dc0cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.971+0000 7fb3daffd640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb3dc15b040 0x7fb3dc0ca740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.40:40952/0 (socket says 172.21.15.40:40952) 2023-10-16T08:38:56.977 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.971+0000 7fb3daffd640 1 -- 172.21.15.40:0/4118542788 learned_addr learned my addr 172.21.15.40:0/4118542788 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:56.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.971+0000 7fb3da7fc640 1 --1- 172.21.15.40:0/4118542788 >> v1:172.21.15.86:6789/0 conn(0x7fb3dc0adec0 0x7fb3dc0c8fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.40:36054/0 (socket says 172.21.15.40:36054) 2023-10-16T08:38:56.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.971+0000 7fb3daffd640 1 -- 172.21.15.40:0/4118542788 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3dc06c7c0 msgr2=0x7fb3dc0cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:56.978 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.971+0000 7fb3daffd640 1 --2- 172.21.15.40:0/4118542788 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3dc06c7c0 0x7fb3dc0cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.971+0000 7fb3daffd640 1 -- 172.21.15.40:0/4118542788 >> v1:172.21.15.86:6789/0 conn(0x7fb3dc0adec0 legacy=0x7fb3dc0c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.971+0000 7fb3daffd640 1 -- 172.21.15.40:0/4118542788 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3dc0cf5b0 con 0x7fb3dc15b040 2023-10-16T08:38:56.979 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.971+0000 7fb3db7fe640 1 --2- 172.21.15.40:0/4118542788 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3dc06c7c0 0x7fb3dc0cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:38:56.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.975+0000 7fb3daffd640 1 --2- 172.21.15.40:0/4118542788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3dc15b040 0x7fb3dc0ca740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009870 tx=0x7fb3cc01c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7877c7eb9def1063 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:56.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.975+0000 7fb3d9ffb640 1 -- 172.21.15.40:0/4118542788 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc0ab600 con 0x7fb3dc15b040 2023-10-16T08:38:56.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.975+0000 7fb3d9ffb640 1 -- 172.21.15.40:0/4118542788 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3cc0a7050 con 0x7fb3dc15b040 2023-10-16T08:38:56.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.975+0000 7fb3d9ffb640 1 -- 172.21.15.40:0/4118542788 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc0ab980 con 0x7fb3dc15b040 2023-10-16T08:38:56.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.975+0000 7fb3e18c0640 1 -- 172.21.15.40:0/4118542788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3dc15b040 msgr2=0x7fb3dc0ca740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.975+0000 7fb3e18c0640 1 --2- 172.21.15.40:0/4118542788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3dc15b040 0x7fb3dc0ca740 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009870 tx=0x7fb3cc01c3f0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.975+0000 7fb3e18c0640 1 -- 172.21.15.40:0/4118542788 shutdown_connections 2023-10-16T08:38:56.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.975+0000 7fb3e18c0640 1 --2- 172.21.15.40:0/4118542788 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3dc06c7c0 0x7fb3dc0cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.975+0000 7fb3e18c0640 1 --2- 172.21.15.40:0/4118542788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3dc15b040 0x7fb3dc0ca740 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.975+0000 7fb3e18c0640 1 -- 172.21.15.40:0/4118542788 >> 172.21.15.40:0/4118542788 conn(0x7fb3dc0ce960 msgr2=0x7fb3dc0ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:56.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.975+0000 7fb3e18c0640 1 -- 172.21.15.40:0/4118542788 shutdown_connections 2023-10-16T08:38:56.982 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.975+0000 7fb3e18c0640 1 -- 172.21.15.40:0/4118542788 wait complete. 2023-10-16T08:38:56.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.975+0000 7fb3e18c0640 1 Processor -- start 2023-10-16T08:38:56.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.975+0000 7fb3e18c0640 1 -- start start 2023-10-16T08:38:56.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3e18c0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3dc06c7c0 0x7fb3dc14fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3e18c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3dc0adec0 0x7fb3dc1553d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3e18c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3dc15b040 0x7fb3dc0b25d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3e18c0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc1f8670 con 0x7fb3dc0adec0 2023-10-16T08:38:56.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3e18c0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc155db0 con 0x7fb3dc15b040 2023-10-16T08:38:56.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3e18c0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3dc1f8930 con 0x7fb3dc06c7c0 2023-10-16T08:38:56.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3da7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3dc0adec0 0x7fb3dc1553d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3da7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3dc0adec0 0x7fb3dc1553d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34124/0 (socket says 172.21.15.40:34124) 2023-10-16T08:38:56.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3da7fc640 1 -- 172.21.15.40:0/453302185 learned_addr learned my addr 172.21.15.40:0/453302185 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:56.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3db7fe640 1 --2- 172.21.15.40:0/453302185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3dc15b040 0x7fb3dc0b25d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3daffd640 1 --2- 172.21.15.40:0/453302185 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3dc06c7c0 0x7fb3dc14fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3da7fc640 1 -- 172.21.15.40:0/453302185 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3dc06c7c0 msgr2=0x7fb3dc14fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3da7fc640 1 --2- 172.21.15.40:0/453302185 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3dc06c7c0 0x7fb3dc14fe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3da7fc640 1 -- 172.21.15.40:0/453302185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3dc15b040 msgr2=0x7fb3dc0b25d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3da7fc640 1 --2- 172.21.15.40:0/453302185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3dc15b040 0x7fb3dc0b25d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3da7fc640 1 -- 172.21.15.40:0/453302185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d4009a70 con 0x7fb3dc0adec0 2023-10-16T08:38:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3db7fe640 1 --2- 172.21.15.40:0/453302185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3dc15b040 0x7fb3dc0b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:38:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3da7fc640 1 --2- 172.21.15.40:0/453302185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3dc0adec0 0x7fb3dc1553d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000c430 tx=0x7fb3d000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3c3fff640 1 -- 172.21.15.40:0/453302185 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d00074e0 con 0x7fb3dc0adec0 2023-10-16T08:38:56.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3c3fff640 1 -- 172.21.15.40:0/453302185 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3d0007640 con 0x7fb3dc0adec0 2023-10-16T08:38:56.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3c3fff640 1 -- 172.21.15.40:0/453302185 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d0004a10 con 0x7fb3dc0adec0 2023-10-16T08:38:56.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3e18c0640 1 -- 172.21.15.40:0/453302185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3cc0094d0 con 0x7fb3dc0adec0 2023-10-16T08:38:56.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3e18c0640 1 -- 172.21.15.40:0/453302185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3dc1f8ef0 con 0x7fb3dc0adec0 2023-10-16T08:38:56.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3e18c0640 1 -- 172.21.15.40:0/453302185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3dc04e9c0 con 0x7fb3dc0adec0 2023-10-16T08:38:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3c3fff640 1 -- 172.21.15.40:0/453302185 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb3d0004b70 con 0x7fb3dc0adec0 2023-10-16T08:38:56.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3c3fff640 1 --2- 172.21.15.40:0/453302185 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb3c80664a0 0x7fb3c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3c3fff640 1 -- 172.21.15.40:0/453302185 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb3d0083f30 con 0x7fb3dc0adec0 2023-10-16T08:38:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.979+0000 7fb3daffd640 1 --2- 172.21.15.40:0/453302185 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb3c80664a0 0x7fb3c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.983+0000 7fb3daffd640 1 --2- 172.21.15.40:0/453302185 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb3c80664a0 0x7fb3c8068960 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0ad000 tx=0x7fb3cc0a6000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:56.991 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:56.983+0000 7fb3c3fff640 1 -- 172.21.15.40:0/453302185 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fb3d00af3d0 con 0x7fb3dc0adec0 2023-10-16T08:38:57.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.175+0000 7fb3e18c0640 1 -- 172.21.15.40:0/453302185 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3dc0ccb20 con 0x7fb3dc0adec0 2023-10-16T08:38:57.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.179+0000 7fb3c3fff640 1 -- 172.21.15.40:0/453302185 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4898 (secure 0 0 0) 0x7fb3d0054eb0 con 0x7fb3dc0adec0 2023-10-16T08:38:57.183 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:57.184 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/32529848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":32529848},{"type":"v1","addr":"172.21.15.86:6837","nonce":32529848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2044121591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2044121591},{"type":"v1","addr":"172.21.15.86:6836","nonce":2044121591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1275484997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1275484997},{"type":"v1","addr":"172.21.15.40:6833","nonce":1275484997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:38:44.984772+0000","modified":"2023-10-16T08:38:47.305255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/148507680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":148507680},{"type":"v1","addr":"172.21.15.40:6835","nonce":148507680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T08:38:57.184 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 6 2023-10-16T08:38:57.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.179+0000 7fb3e18c0640 1 -- 172.21.15.40:0/453302185 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb3c80664a0 msgr2=0x7fb3c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:57.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.179+0000 7fb3e18c0640 1 --2- 172.21.15.40:0/453302185 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb3c80664a0 0x7fb3c8068960 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc0ad000 tx=0x7fb3cc0a6000 comp rx=0 tx=0).stop 2023-10-16T08:38:57.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.179+0000 7fb3e18c0640 1 -- 172.21.15.40:0/453302185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3dc0adec0 msgr2=0x7fb3dc1553d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:57.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.179+0000 7fb3e18c0640 1 --2- 172.21.15.40:0/453302185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3dc0adec0 0x7fb3dc1553d0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fb3d000c430 tx=0x7fb3d000c8f0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.179+0000 7fb3e18c0640 1 -- 172.21.15.40:0/453302185 shutdown_connections 2023-10-16T08:38:57.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.179+0000 7fb3e18c0640 1 --2- 172.21.15.40:0/453302185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3dc15b040 0x7fb3dc0b25d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.179+0000 7fb3e18c0640 1 --2- 172.21.15.40:0/453302185 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb3c80664a0 0x7fb3c8068960 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.179+0000 7fb3e18c0640 1 --2- 172.21.15.40:0/453302185 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb3dc0adec0 0x7fb3dc1553d0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.179+0000 7fb3e18c0640 1 --2- 172.21.15.40:0/453302185 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb3dc06c7c0 0x7fb3dc14fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.179+0000 7fb3e18c0640 1 -- 172.21.15.40:0/453302185 >> 172.21.15.40:0/453302185 conn(0x7fb3dc0ce960 msgr2=0x7fb3dc1561d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:57.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.179+0000 7fb3e18c0640 1 -- 172.21.15.40:0/453302185 shutdown_connections 2023-10-16T08:38:57.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.179+0000 7fb3e18c0640 1 -- 172.21.15.40:0/453302185 wait complete. 2023-10-16T08:38:57.202 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:38:57.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.303+0000 7ff1b02b5640 1 Processor -- start 2023-10-16T08:38:57.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1b02b5640 1 -- start start 2023-10-16T08:38:57.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1b02b5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1a815b040 0x7ff1a80ca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:57.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1b02b5640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a80adca0 con 0x7ff1a806c7c0 2023-10-16T08:38:57.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1b02b5640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a80c5840 con 0x7ff1a80adec0 2023-10-16T08:38:57.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1b02b5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a80c9dd0 con 0x7ff1a815b040 2023-10-16T08:38:57.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ae852640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff1a806c7c0 0x7ff1a80c96c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50424/0 (socket says 172.21.15.40:50424) 2023-10-16T08:38:57.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ae852640 1 -- 172.21.15.40:0/1221151381 learned_addr learned my addr 172.21.15.40:0/1221151381 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:57.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ae051640 1 --2- 172.21.15.40:0/1221151381 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1a815b040 0x7ff1a80ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:57.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ad04f640 1 -- 172.21.15.40:0/1221151381 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2986832726 0 0) 0x7ff1a80adca0 con 0x7ff1a806c7c0 2023-10-16T08:38:57.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ad04f640 1 -- 172.21.15.40:0/1221151381 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff194003530 con 0x7ff1a806c7c0 2023-10-16T08:38:57.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ad04f640 1 -- 172.21.15.40:0/1221151381 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2653455205 0 0) 0x7ff194003530 con 0x7ff1a806c7c0 2023-10-16T08:38:57.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ad04f640 1 -- 172.21.15.40:0/1221151381 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1a815b040 msgr2=0x7ff1a80ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:57.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ad04f640 1 --2- 172.21.15.40:0/1221151381 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1a815b040 0x7ff1a80ca740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ad04f640 1 -- 172.21.15.40:0/1221151381 >> v1:172.21.15.86:6789/0 conn(0x7ff1a80adec0 legacy=0x7ff1a80c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:57.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ad04f640 1 -- 172.21.15.40:0/1221151381 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a80cf890 con 0x7ff1a806c7c0 2023-10-16T08:38:57.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ad04f640 1 -- 172.21.15.40:0/1221151381 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7ff1a0001a00 con 0x7ff1a806c7c0 2023-10-16T08:38:57.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ad04f640 1 -- 172.21.15.40:0/1221151381 >> v1:172.21.15.40:6789/0 conn(0x7ff1a806c7c0 legacy=0x7ff1a80c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:57.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ad04f640 1 --2- 172.21.15.40:0/1221151381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff194003a20 0x7ff194003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:57.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ad04f640 1 --2- 172.21.15.40:0/1221151381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1940047d0 0x7ff194004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:57.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ad04f640 1 --2- 172.21.15.40:0/1221151381 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1a815b040 0x7ff19400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:57.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ad04f640 1 -- 172.21.15.40:0/1221151381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a80c9dd0 con 0x7ff194003a20 2023-10-16T08:38:57.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ad04f640 1 -- 172.21.15.40:0/1221151381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1940054d0 con 0x7ff1940047d0 2023-10-16T08:38:57.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ad04f640 1 -- 172.21.15.40:0/1221151381 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff194005690 con 0x7ff1a815b040 2023-10-16T08:38:57.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ae051640 1 --2- 172.21.15.40:0/1221151381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff194003a20 0x7ff194003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:57.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ae852640 1 --2- 172.21.15.40:0/1221151381 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1a815b040 0x7ff19400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:57.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ad850640 1 --2- 172.21.15.40:0/1221151381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1940047d0 0x7ff194004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:57.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ae051640 1 -- 172.21.15.40:0/1221151381 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1a815b040 msgr2=0x7ff19400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:57.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ae051640 1 --2- 172.21.15.40:0/1221151381 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1a815b040 0x7ff19400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ae051640 1 -- 172.21.15.40:0/1221151381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1940047d0 msgr2=0x7ff194004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:57.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ae051640 1 --2- 172.21.15.40:0/1221151381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1940047d0 0x7ff194004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ae051640 1 -- 172.21.15.40:0/1221151381 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff194003530 con 0x7ff194003a20 2023-10-16T08:38:57.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.307+0000 7ff1ad850640 1 --2- 172.21.15.40:0/1221151381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1940047d0 0x7ff194004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:57.322 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1ae051640 1 --2- 172.21.15.40:0/1221151381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff194003a20 0x7ff194003e40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009ae0 tx=0x7ff1a401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d319b97b2f5cf7a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:57.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1ad04f640 1 -- 172.21.15.40:0/1221151381 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a40a6510 con 0x7ff194003a20 2023-10-16T08:38:57.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1ad04f640 1 -- 172.21.15.40:0/1221151381 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a40a6670 con 0x7ff194003a20 2023-10-16T08:38:57.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1ad04f640 1 -- 172.21.15.40:0/1221151381 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a40a6a20 con 0x7ff194003a20 2023-10-16T08:38:57.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1b02b5640 1 -- 172.21.15.40:0/1221151381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff194003a20 msgr2=0x7ff194003e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:57.323 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1b02b5640 1 --2- 172.21.15.40:0/1221151381 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff194003a20 0x7ff194003e40 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff1a4009ae0 tx=0x7ff1a401ca20 comp rx=0 tx=0).stop 2023-10-16T08:38:57.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1ae852640 1 -- 172.21.15.40:0/1221151381 reap_dead start 2023-10-16T08:38:57.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1b02b5640 1 -- 172.21.15.40:0/1221151381 shutdown_connections 2023-10-16T08:38:57.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1b02b5640 1 -- 172.21.15.40:0/1221151381 >> 172.21.15.40:0/1221151381 conn(0x7ff1a80ce960 msgr2=0x7ff1a80ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:57.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1b02b5640 1 -- 172.21.15.40:0/1221151381 shutdown_connections 2023-10-16T08:38:57.324 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1b02b5640 1 -- 172.21.15.40:0/1221151381 wait complete. 2023-10-16T08:38:57.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1b02b5640 1 Processor -- start 2023-10-16T08:38:57.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1b02b5640 1 -- start start 2023-10-16T08:38:57.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1b02b5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1a814afd0 0x7ff1a80c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:57.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1b02b5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1a814f750 0x7ff1a80c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:57.325 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1b02b5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1a815b4b0 0x7ff1a815b930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:57.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1b02b5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a815be70 con 0x7ff1a815b4b0 2023-10-16T08:38:57.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1b02b5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a814b3b0 con 0x7ff1a814f750 2023-10-16T08:38:57.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1b02b5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a815d880 con 0x7ff1a814afd0 2023-10-16T08:38:57.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1ae852640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1a815b4b0 0x7ff1a815b930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:57.326 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1ae852640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1a815b4b0 0x7ff1a815b930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34138/0 (socket says 172.21.15.40:34138) 2023-10-16T08:38:57.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1ae852640 1 -- 172.21.15.40:0/1957191193 learned_addr learned my addr 172.21.15.40:0/1957191193 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:57.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1ae051640 1 --2- 172.21.15.40:0/1957191193 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1a814afd0 0x7ff1a80c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:57.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1ad850640 1 --2- 172.21.15.40:0/1957191193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1a814f750 0x7ff1a80c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:57.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1ae852640 1 -- 172.21.15.40:0/1957191193 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1a814afd0 msgr2=0x7ff1a80c5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:57.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1ae852640 1 --2- 172.21.15.40:0/1957191193 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1a814afd0 0x7ff1a80c5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1ae852640 1 -- 172.21.15.40:0/1957191193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1a814f750 msgr2=0x7ff1a80c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:57.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1ae852640 1 --2- 172.21.15.40:0/1957191193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1a814f750 0x7ff1a80c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1ae852640 1 -- 172.21.15.40:0/1957191193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a80c6310 con 0x7ff1a815b4b0 2023-10-16T08:38:57.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1ad850640 1 --2- 172.21.15.40:0/1957191193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1a814f750 0x7ff1a80c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:57.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1ae852640 1 --2- 172.21.15.40:0/1957191193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1a815b4b0 0x7ff1a815b930 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff1a806f470 tx=0x7ff1a000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:57.328 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff19effd640 1 -- 172.21.15.40:0/1957191193 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0007a90 con 0x7ff1a815b4b0 2023-10-16T08:38:57.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff19effd640 1 -- 172.21.15.40:0/1957191193 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a000edc0 con 0x7ff1a815b4b0 2023-10-16T08:38:57.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1b02b5640 1 -- 172.21.15.40:0/1957191193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a80c65f0 con 0x7ff1a815b4b0 2023-10-16T08:38:57.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff19effd640 1 -- 172.21.15.40:0/1957191193 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a0013660 con 0x7ff1a815b4b0 2023-10-16T08:38:57.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.311+0000 7ff1b02b5640 1 -- 172.21.15.40:0/1957191193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a80c5f70 con 0x7ff1a815b4b0 2023-10-16T08:38:57.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.315+0000 7ff1b02b5640 1 -- 172.21.15.40:0/1957191193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a804e9c0 con 0x7ff1a815b4b0 2023-10-16T08:38:57.329 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.315+0000 7ff19effd640 1 -- 172.21.15.40:0/1957191193 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff1a00137c0 con 0x7ff1a815b4b0 2023-10-16T08:38:57.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.315+0000 7ff19effd640 1 --2- 172.21.15.40:0/1957191193 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff18c0664a0 0x7ff18c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:57.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.315+0000 7ff19effd640 1 -- 172.21.15.40:0/1957191193 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff1a0084de0 con 0x7ff1a815b4b0 2023-10-16T08:38:57.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.315+0000 7ff1ae051640 1 --2- 172.21.15.40:0/1957191193 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff18c0664a0 0x7ff18c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:57.330 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.315+0000 7ff1ae051640 1 --2- 172.21.15.40:0/1957191193 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff18c0664a0 0x7ff18c068960 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40af000 tx=0x7ff1a4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:57.331 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.319+0000 7ff19effd640 1 -- 172.21.15.40:0/1957191193 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7ff1a0052340 con 0x7ff1a815b4b0 2023-10-16T08:38:57.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.471+0000 7ff1b02b5640 1 -- 172.21.15.40:0/1957191193 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1a815d9c0 con 0x7ff1a815b4b0 2023-10-16T08:38:57.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.471+0000 7ff19effd640 1 -- 172.21.15.40:0/1957191193 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16445 (secure 0 0 0) 0x7ff1a0055960 con 0x7ff1a815b4b0 2023-10-16T08:38:57.478 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:57.479 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:46.611098+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:38:57.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.475+0000 7ff1b02b5640 1 -- 172.21.15.40:0/1957191193 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff18c0664a0 msgr2=0x7ff18c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:57.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.475+0000 7ff1b02b5640 1 --2- 172.21.15.40:0/1957191193 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff18c0664a0 0x7ff18c068960 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff1a40af000 tx=0x7ff1a4008040 comp rx=0 tx=0).stop 2023-10-16T08:38:57.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.475+0000 7ff1b02b5640 1 -- 172.21.15.40:0/1957191193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1a815b4b0 msgr2=0x7ff1a815b930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:57.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.475+0000 7ff1b02b5640 1 --2- 172.21.15.40:0/1957191193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1a815b4b0 0x7ff1a815b930 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ff1a806f470 tx=0x7ff1a000e530 comp rx=0 tx=0).stop 2023-10-16T08:38:57.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.475+0000 7ff1b02b5640 1 -- 172.21.15.40:0/1957191193 shutdown_connections 2023-10-16T08:38:57.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.475+0000 7ff1b02b5640 1 --2- 172.21.15.40:0/1957191193 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff18c0664a0 0x7ff18c068960 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.475+0000 7ff1b02b5640 1 --2- 172.21.15.40:0/1957191193 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff1a815b4b0 0x7ff1a815b930 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.475+0000 7ff1b02b5640 1 --2- 172.21.15.40:0/1957191193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff1a814f750 0x7ff1a80c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.475+0000 7ff1b02b5640 1 --2- 172.21.15.40:0/1957191193 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff1a814afd0 0x7ff1a80c5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.475+0000 7ff1b02b5640 1 -- 172.21.15.40:0/1957191193 >> 172.21.15.40:0/1957191193 conn(0x7ff1a80ce960 msgr2=0x7ff1a80c9500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:57.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.475+0000 7ff1b02b5640 1 -- 172.21.15.40:0/1957191193 shutdown_connections 2023-10-16T08:38:57.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.475+0000 7ff1b02b5640 1 -- 172.21.15.40:0/1957191193 wait complete. 2023-10-16T08:38:57.497 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:38:57.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.603+0000 7f985df16640 1 Processor -- start 2023-10-16T08:38:57.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.603+0000 7f985df16640 1 -- start start 2023-10-16T08:38:57.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.603+0000 7f985df16640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f98581648d0 0x7f98580cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:57.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.603+0000 7f985df16640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f985806c550 0x7f98580ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:57.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.603+0000 7f985df16640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9858165c90 con 0x7f98580c15f0 2023-10-16T08:38:57.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.603+0000 7f985df16640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9858165080 con 0x7f985806c550 2023-10-16T08:38:57.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.603+0000 7f985df16640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9858165360 con 0x7f98581648d0 2023-10-16T08:38:57.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.603+0000 7f9857fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f985806c550 0x7f98580ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:57.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.603+0000 7f9857fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f985806c550 0x7f98580ce1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34140/0 (socket says 172.21.15.40:34140) 2023-10-16T08:38:57.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.603+0000 7f9857fff640 1 -- 172.21.15.40:0/4094652637 learned_addr learned my addr 172.21.15.40:0/4094652637 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:57.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.603+0000 7f98577fe640 1 --2- 172.21.15.40:0/4094652637 >> v2:172.21.15.86:3300/0 conn(0x7f98581648d0 0x7f98580cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:57.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.603+0000 7f9857fff640 1 -- 172.21.15.40:0/4094652637 >> v2:172.21.15.86:3300/0 conn(0x7f98581648d0 msgr2=0x7f98580cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:57.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.603+0000 7f9857fff640 1 --2- 172.21.15.40:0/4094652637 >> v2:172.21.15.86:3300/0 conn(0x7f98581648d0 0x7f98580cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.603+0000 7f9857fff640 1 -- 172.21.15.40:0/4094652637 >> v1:172.21.15.86:6789/0 conn(0x7f98580c15f0 legacy=0x7f98580cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:57.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.603+0000 7f9857fff640 1 -- 172.21.15.40:0/4094652637 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9858165640 con 0x7f985806c550 2023-10-16T08:38:57.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f9857fff640 1 --2- 172.21.15.40:0/4094652637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985806c550 0x7f98580ce1e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9858071da0 tx=0x7f985001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a495bbe8b27300b8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:57.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f98567fc640 1 -- 172.21.15.40:0/4094652637 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98500a56a0 con 0x7f985806c550 2023-10-16T08:38:57.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f98567fc640 1 -- 172.21.15.40:0/4094652637 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98500a5800 con 0x7f985806c550 2023-10-16T08:38:57.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f98567fc640 1 -- 172.21.15.40:0/4094652637 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98500a5b90 con 0x7f985806c550 2023-10-16T08:38:57.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f985df16640 1 -- 172.21.15.40:0/4094652637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985806c550 msgr2=0x7f98580ce1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:57.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f985df16640 1 --2- 172.21.15.40:0/4094652637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985806c550 0x7f98580ce1e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9858071da0 tx=0x7f985001c8d0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f985df16640 1 -- 172.21.15.40:0/4094652637 shutdown_connections 2023-10-16T08:38:57.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f985df16640 1 --2- 172.21.15.40:0/4094652637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985806c550 0x7f98580ce1e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f985df16640 1 --2- 172.21.15.40:0/4094652637 >> v2:172.21.15.86:3300/0 conn(0x7f98581648d0 0x7f98580cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f985df16640 1 -- 172.21.15.40:0/4094652637 >> 172.21.15.40:0/4094652637 conn(0x7f98580ce960 msgr2=0x7f98580ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:57.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f985df16640 1 -- 172.21.15.40:0/4094652637 shutdown_connections 2023-10-16T08:38:57.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f985df16640 1 -- 172.21.15.40:0/4094652637 wait complete. 2023-10-16T08:38:57.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f985df16640 1 Processor -- start 2023-10-16T08:38:57.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f985df16640 1 -- start start 2023-10-16T08:38:57.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f985df16640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985806c550 0x7f9858164020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:57.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f985df16640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98580c15f0 0x7f98580b3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:57.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f985df16640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98581648d0 0x7f985814b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:57.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f985df16640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f985814bd20 con 0x7f985806c550 2023-10-16T08:38:57.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f985df16640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9858163010 con 0x7f98581648d0 2023-10-16T08:38:57.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f985df16640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9858163260 con 0x7f98580c15f0 2023-10-16T08:38:57.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f98577fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985806c550 0x7f9858164020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:57.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f98577fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985806c550 0x7f9858164020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34146/0 (socket says 172.21.15.40:34146) 2023-10-16T08:38:57.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f98577fe640 1 -- 172.21.15.40:0/2222014525 learned_addr learned my addr 172.21.15.40:0/2222014525 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:57.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f9857fff640 1 --2- 172.21.15.40:0/2222014525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98581648d0 0x7f985814b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:57.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f9856ffd640 1 --2- 172.21.15.40:0/2222014525 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98580c15f0 0x7f98580b3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:57.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f98577fe640 1 -- 172.21.15.40:0/2222014525 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98580c15f0 msgr2=0x7f98580b3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:57.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f98577fe640 1 --2- 172.21.15.40:0/2222014525 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98580c15f0 0x7f98580b3ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f98577fe640 1 -- 172.21.15.40:0/2222014525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98581648d0 msgr2=0x7f985814b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:57.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f98577fe640 1 --2- 172.21.15.40:0/2222014525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98581648d0 0x7f985814b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f98577fe640 1 -- 172.21.15.40:0/2222014525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9850009b30 con 0x7f985806c550 2023-10-16T08:38:57.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f98577fe640 1 --2- 172.21.15.40:0/2222014525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985806c550 0x7f9858164020 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f984800d800 tx=0x7f984800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:57.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f984ffff640 1 -- 172.21.15.40:0/2222014525 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984800b6c0 con 0x7f985806c550 2023-10-16T08:38:57.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f984ffff640 1 -- 172.21.15.40:0/2222014525 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f984800b820 con 0x7f985806c550 2023-10-16T08:38:57.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f985df16640 1 -- 172.21.15.40:0/2222014525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9858163540 con 0x7f985806c550 2023-10-16T08:38:57.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f984ffff640 1 -- 172.21.15.40:0/2222014525 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f984800bb90 con 0x7f985806c550 2023-10-16T08:38:57.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.607+0000 7f985df16640 1 -- 172.21.15.40:0/2222014525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9858163b00 con 0x7f985806c550 2023-10-16T08:38:57.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.611+0000 7f985df16640 1 -- 172.21.15.40:0/2222014525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f985804e9c0 con 0x7f985806c550 2023-10-16T08:38:57.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.611+0000 7f984ffff640 1 -- 172.21.15.40:0/2222014525 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f984800bcf0 con 0x7f985806c550 2023-10-16T08:38:57.621 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.611+0000 7f984ffff640 1 --2- 172.21.15.40:0/2222014525 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f98380664a0 0x7f9838068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:57.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.611+0000 7f984ffff640 1 -- 172.21.15.40:0/2222014525 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98480843c0 con 0x7f985806c550 2023-10-16T08:38:57.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.611+0000 7f9856ffd640 1 --2- 172.21.15.40:0/2222014525 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f98380664a0 0x7f9838068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:57.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.611+0000 7f9856ffd640 1 --2- 172.21.15.40:0/2222014525 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f98380664a0 0x7f9838068960 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f98580add20 tx=0x7f9840009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:57.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.615+0000 7f984ffff640 1 -- 172.21.15.40:0/2222014525 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f98480518d0 con 0x7f985806c550 2023-10-16T08:38:57.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.803+0000 7f985df16640 1 -- 172.21.15.40:0/2222014525 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f98580715e0 con 0x7f985806c550 2023-10-16T08:38:57.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.803+0000 7f984ffff640 1 -- 172.21.15.40:0/2222014525 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4898 (secure 0 0 0) 0x7f9848054ef0 con 0x7f985806c550 2023-10-16T08:38:57.808 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:38:57.808 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/32529848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":32529848},{"type":"v1","addr":"172.21.15.86:6837","nonce":32529848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2044121591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2044121591},{"type":"v1","addr":"172.21.15.86:6836","nonce":2044121591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1275484997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1275484997},{"type":"v1","addr":"172.21.15.40:6833","nonce":1275484997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:38:44.984772+0000","modified":"2023-10-16T08:38:47.305255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/148507680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":148507680},{"type":"v1","addr":"172.21.15.40:6835","nonce":148507680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T08:38:57.809 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 6 2023-10-16T08:38:57.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.803+0000 7f985df16640 1 -- 172.21.15.40:0/2222014525 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f98380664a0 msgr2=0x7f9838068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:57.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.803+0000 7f985df16640 1 --2- 172.21.15.40:0/2222014525 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f98380664a0 0x7f9838068960 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f98580add20 tx=0x7f9840009040 comp rx=0 tx=0).stop 2023-10-16T08:38:57.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.803+0000 7f985df16640 1 -- 172.21.15.40:0/2222014525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985806c550 msgr2=0x7f9858164020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:57.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.803+0000 7f985df16640 1 --2- 172.21.15.40:0/2222014525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985806c550 0x7f9858164020 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f984800d800 tx=0x7f984800dcc0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.803+0000 7f985df16640 1 -- 172.21.15.40:0/2222014525 shutdown_connections 2023-10-16T08:38:57.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.803+0000 7f985df16640 1 --2- 172.21.15.40:0/2222014525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98581648d0 0x7f985814b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.803+0000 7f985df16640 1 --2- 172.21.15.40:0/2222014525 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98580c15f0 0x7f98580b3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.803+0000 7f985df16640 1 --2- 172.21.15.40:0/2222014525 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f98380664a0 0x7f9838068960 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.803+0000 7f985df16640 1 --2- 172.21.15.40:0/2222014525 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f985806c550 0x7f9858164020 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:57.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.803+0000 7f985df16640 1 -- 172.21.15.40:0/2222014525 >> 172.21.15.40:0/2222014525 conn(0x7f98580ce960 msgr2=0x7f98580ae3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:57.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.803+0000 7f985df16640 1 -- 172.21.15.40:0/2222014525 shutdown_connections 2023-10-16T08:38:57.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:57.803+0000 7f985df16640 1 -- 172.21.15.40:0/2222014525 wait complete. 2023-10-16T08:38:58.227 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 6, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-16T08:38:44.984772+0000', 'modified': '2023-10-16T08:38:47.305255+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4287}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4287': {'gid': 4287, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.40:6835/148507680', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.40:6834', 'nonce': 148507680}, {'type': 'v1', 'addr': '172.21.15.40:6835', 'nonce': 148507680}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-10-16T08:38:58.228 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-16T08:38:58.228 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-16T08:38:58.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.355+0000 7fa230705640 1 Processor -- start 2023-10-16T08:38:58.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.355+0000 7fa230705640 1 -- start start 2023-10-16T08:38:58.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.355+0000 7fa230705640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa22815cf40 0x7fa2280ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.355+0000 7fa230705640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa2280ade00 0x7fa2280ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.355+0000 7fa230705640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa22806b540 con 0x7fa22815d320 2023-10-16T08:38:58.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.355+0000 7fa230705640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa22806fbf0 con 0x7fa2280ade00 2023-10-16T08:38:58.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.355+0000 7fa230705640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22806b7e0 con 0x7fa22815cf40 2023-10-16T08:38:58.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.355+0000 7fa22eca2640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa2280ade00 0x7fa2280ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.355+0000 7fa22e4a1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa22815cf40 0x7fa2280ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.355+0000 7fa22e4a1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa22815cf40 0x7fa2280ccef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.40:46998/0 (socket says 172.21.15.40:46998) 2023-10-16T08:38:58.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.355+0000 7fa22dca0640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fa22815d320 0x7fa22806f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.40:36074/0 (socket says 172.21.15.40:36074) 2023-10-16T08:38:58.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.355+0000 7fa22eca2640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa2280ade00 0x7fa2280ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.40:41000/0 (socket says 172.21.15.40:41000) 2023-10-16T08:38:58.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.355+0000 7fa22e4a1640 1 -- 172.21.15.40:0/416075322 learned_addr learned my addr 172.21.15.40:0/416075322 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:58.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.359+0000 7fa22e4a1640 1 -- 172.21.15.40:0/416075322 >> v2:172.21.15.86:3300/0 conn(0x7fa2280ade00 msgr2=0x7fa2280ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:58.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.359+0000 7fa22e4a1640 1 --2- 172.21.15.40:0/416075322 >> v2:172.21.15.86:3300/0 conn(0x7fa2280ade00 0x7fa2280ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.359+0000 7fa22e4a1640 1 -- 172.21.15.40:0/416075322 >> v1:172.21.15.86:6789/0 conn(0x7fa22815d320 legacy=0x7fa22806f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:58.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.359+0000 7fa22e4a1640 1 -- 172.21.15.40:0/416075322 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa22806bac0 con 0x7fa22815cf40 2023-10-16T08:38:58.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.359+0000 7fa22eca2640 1 --2- 172.21.15.40:0/416075322 >> v2:172.21.15.86:3300/0 conn(0x7fa2280ade00 0x7fa2280ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:38:58.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.359+0000 7fa22e4a1640 1 --2- 172.21.15.40:0/416075322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa22815cf40 0x7fa2280ccef0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa218009870 tx=0x7fa21801c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=693308ca06419c8b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:58.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.359+0000 7fa22d49f640 1 -- 172.21.15.40:0/416075322 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2180ab620 con 0x7fa22815cf40 2023-10-16T08:38:58.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.359+0000 7fa22d49f640 1 -- 172.21.15.40:0/416075322 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2180ab780 con 0x7fa22815cf40 2023-10-16T08:38:58.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.359+0000 7fa22d49f640 1 -- 172.21.15.40:0/416075322 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2180abb10 con 0x7fa22815cf40 2023-10-16T08:38:58.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.359+0000 7fa230705640 1 -- 172.21.15.40:0/416075322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa22815cf40 msgr2=0x7fa2280ccef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:58.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.359+0000 7fa230705640 1 --2- 172.21.15.40:0/416075322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa22815cf40 0x7fa2280ccef0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa218009870 tx=0x7fa21801c510 comp rx=0 tx=0).stop 2023-10-16T08:38:58.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa230705640 1 -- 172.21.15.40:0/416075322 shutdown_connections 2023-10-16T08:38:58.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa230705640 1 --2- 172.21.15.40:0/416075322 >> v2:172.21.15.86:3300/0 conn(0x7fa2280ade00 0x7fa2280ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa230705640 1 --2- 172.21.15.40:0/416075322 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa22815cf40 0x7fa2280ccef0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa230705640 1 -- 172.21.15.40:0/416075322 >> 172.21.15.40:0/416075322 conn(0x7fa2280ce960 msgr2=0x7fa2280ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:58.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa230705640 1 -- 172.21.15.40:0/416075322 shutdown_connections 2023-10-16T08:38:58.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa230705640 1 -- 172.21.15.40:0/416075322 wait complete. 2023-10-16T08:38:58.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa230705640 1 Processor -- start 2023-10-16T08:38:58.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa230705640 1 -- start start 2023-10-16T08:38:58.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa230705640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2280ade00 0x7fa2280b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa230705640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa22815cf40 0x7fa22814a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa230705640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22815d320 0x7fa2281f93f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa230705640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22815c820 con 0x7fa2280ade00 2023-10-16T08:38:58.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa230705640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa22815bfa0 con 0x7fa22815d320 2023-10-16T08:38:58.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa230705640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa22815c260 con 0x7fa22815cf40 2023-10-16T08:38:58.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa22e4a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2280ade00 0x7fa2280b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa22e4a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2280ade00 0x7fa2280b1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34158/0 (socket says 172.21.15.40:34158) 2023-10-16T08:38:58.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa22e4a1640 1 -- 172.21.15.40:0/2011686016 learned_addr learned my addr 172.21.15.40:0/2011686016 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:58.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa22e4a1640 1 -- 172.21.15.40:0/2011686016 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa22815cf40 msgr2=0x7fa22814a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:58.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa22dca0640 1 --2- 172.21.15.40:0/2011686016 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa22815cf40 0x7fa22814a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa22eca2640 1 --2- 172.21.15.40:0/2011686016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22815d320 0x7fa2281f93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa22e4a1640 1 --2- 172.21.15.40:0/2011686016 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa22815cf40 0x7fa22814a930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa22e4a1640 1 -- 172.21.15.40:0/2011686016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22815d320 msgr2=0x7fa2281f93f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:58.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa22e4a1640 1 --2- 172.21.15.40:0/2011686016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22815d320 0x7fa2281f93f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa22e4a1640 1 -- 172.21.15.40:0/2011686016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa220009a70 con 0x7fa2280ade00 2023-10-16T08:38:58.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa22eca2640 1 --2- 172.21.15.40:0/2011686016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22815d320 0x7fa2281f93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:38:58.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa22e4a1640 1 --2- 172.21.15.40:0/2011686016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2280ade00 0x7fa2280b1580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa21801cdb0 tx=0x7fa21801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:58.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa21effd640 1 -- 172.21.15.40:0/2011686016 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2180abe80 con 0x7fa2280ade00 2023-10-16T08:38:58.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa21effd640 1 -- 172.21.15.40:0/2011686016 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2180be430 con 0x7fa2280ade00 2023-10-16T08:38:58.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa230705640 1 -- 172.21.15.40:0/2011686016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2180094d0 con 0x7fa2280ade00 2023-10-16T08:38:58.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa21effd640 1 -- 172.21.15.40:0/2011686016 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2180be7c0 con 0x7fa2280ade00 2023-10-16T08:38:58.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.363+0000 7fa230705640 1 -- 172.21.15.40:0/2011686016 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22815a050 con 0x7fa2280ade00 2023-10-16T08:38:58.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa21effd640 1 -- 172.21.15.40:0/2011686016 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa2180be920 con 0x7fa2280ade00 2023-10-16T08:38:58.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa21effd640 1 --2- 172.21.15.40:0/2011686016 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa20c0664a0 0x7fa20c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa21effd640 1 -- 172.21.15.40:0/2011686016 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa21811e120 con 0x7fa2280ade00 2023-10-16T08:38:58.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa22dca0640 1 --2- 172.21.15.40:0/2011686016 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa20c0664a0 0x7fa20c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa23384d000 1 Processor -- start 2023-10-16T08:38:58.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa23384d000 1 -- start start 2023-10-16T08:38:58.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa22dca0640 1 --2- 172.21.15.40:0/2011686016 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa20c0664a0 0x7fa20c068960 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa2281555f0 tx=0x7fa224009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:58.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa23384d000 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f88a0 0x56367b1f8c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa23384d000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56367b1f91c0 0x56367b1ae710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa23384d000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367b1aec50 0x56367b1af030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa23384d000 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367afa3cc0 con 0x56367b1f88a0 2023-10-16T08:38:58.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa23384d000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367b006920 con 0x56367b1aec50 2023-10-16T08:38:58.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa23384d000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x56367b003e50 con 0x56367b1f91c0 2023-10-16T08:38:58.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa22e4a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f88a0 0x56367b1f8c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa22e4a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f88a0 0x56367b1f8c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34170/0 (socket says 172.21.15.40:34170) 2023-10-16T08:38:58.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa22e4a1640 1 -- 172.21.15.40:0/2125557829 learned_addr learned my addr 172.21.15.40:0/2125557829 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:58.384 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa22eca2640 1 --2- 172.21.15.40:0/2125557829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367b1aec50 0x56367b1af030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa22dca0640 1 --2- 172.21.15.40:0/2125557829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56367b1f91c0 0x56367b1ae710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa22e4a1640 1 -- 172.21.15.40:0/2125557829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56367b1f91c0 msgr2=0x56367b1ae710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:58.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa22e4a1640 1 --2- 172.21.15.40:0/2125557829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56367b1f91c0 0x56367b1ae710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa22e4a1640 1 -- 172.21.15.40:0/2125557829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367b1aec50 msgr2=0x56367b1af030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:58.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa22e4a1640 1 --2- 172.21.15.40:0/2125557829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367b1aec50 0x56367b1af030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa22e4a1640 1 -- 172.21.15.40:0/2125557829 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56367acf8890 con 0x56367b1f88a0 2023-10-16T08:38:58.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa22e4a1640 1 --2- 172.21.15.40:0/2125557829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f88a0 0x56367b1f8c80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa2180eb760 tx=0x7fa2180ef960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8c80a81c742a97 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:58.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa20bfff640 1 -- 172.21.15.40:0/2125557829 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2180f33b0 con 0x56367b1f88a0 2023-10-16T08:38:58.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa20bfff640 1 -- 172.21.15.40:0/2125557829 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2180f7560 con 0x56367b1f88a0 2023-10-16T08:38:58.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa20bfff640 1 -- 172.21.15.40:0/2125557829 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2180f3160 con 0x56367b1f88a0 2023-10-16T08:38:58.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa23384d000 1 -- 172.21.15.40:0/2125557829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f88a0 msgr2=0x56367b1f8c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:58.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa23384d000 1 --2- 172.21.15.40:0/2125557829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f88a0 0x56367b1f8c80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fa2180eb760 tx=0x7fa2180ef960 comp rx=0 tx=0).stop 2023-10-16T08:38:58.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa23384d000 1 -- 172.21.15.40:0/2125557829 shutdown_connections 2023-10-16T08:38:58.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa23384d000 1 --2- 172.21.15.40:0/2125557829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367b1aec50 0x56367b1af030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa23384d000 1 --2- 172.21.15.40:0/2125557829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56367b1f91c0 0x56367b1ae710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa23384d000 1 --2- 172.21.15.40:0/2125557829 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f88a0 0x56367b1f8c80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.367+0000 7fa23384d000 1 -- 172.21.15.40:0/2125557829 >> 172.21.15.40:0/2125557829 conn(0x56367b1a5fc0 msgr2=0x56367b1acff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:58.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 1 -- 172.21.15.40:0/2125557829 shutdown_connections 2023-10-16T08:38:58.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 1 -- 172.21.15.40:0/2125557829 wait complete. 2023-10-16T08:38:58.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 1 -- start start 2023-10-16T08:38:58.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 1 Processor -- start 2023-10-16T08:38:58.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56367b1aec50 0x56367b1b9860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367b1f88a0 0x56367b1b9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f91c0 0x56367b1ba2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367afff740 con 0x56367b1f91c0 2023-10-16T08:38:58.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367afbed10 con 0x56367b1f88a0 2023-10-16T08:38:58.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x56367afbe9b0 con 0x56367b1aec50 2023-10-16T08:38:58.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 10 client.0 _ll_register_callbacks cb 0x56367b19a390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T08:38:58.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa22eca2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f91c0 0x56367b1ba2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa22eca2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f91c0 0x56367b1ba2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34184/0 (socket says 172.21.15.40:34184) 2023-10-16T08:38:58.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa22eca2640 1 -- 172.21.15.40:0/3681881551 learned_addr learned my addr 172.21.15.40:0/3681881551 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:58.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa22dca0640 1 --2- 172.21.15.40:0/3681881551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367b1f88a0 0x56367b1b9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa22e4a1640 1 --2- 172.21.15.40:0/3681881551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56367b1aec50 0x56367b1b9860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa22eca2640 1 -- 172.21.15.40:0/3681881551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56367b1aec50 msgr2=0x56367b1b9860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:58.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa22eca2640 1 --2- 172.21.15.40:0/3681881551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56367b1aec50 0x56367b1b9860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa22eca2640 1 -- 172.21.15.40:0/3681881551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367b1f88a0 msgr2=0x56367b1b9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:58.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa22eca2640 1 --2- 172.21.15.40:0/3681881551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367b1f88a0 0x56367b1b9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa22eca2640 1 -- 172.21.15.40:0/3681881551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22815a050 con 0x56367b1f91c0 2023-10-16T08:38:58.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa22eca2640 1 --2- 172.21.15.40:0/3681881551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f91c0 0x56367b1ba2e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa220007d10 tx=0x7fa220010590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b1e9cfb0130bf9 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:58.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa2097fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-16T08:38:58.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa2097fa640 1 -- 172.21.15.40:0/3681881551 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22000e050 con 0x56367b1f91c0 2023-10-16T08:38:58.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 1 -- 172.21.15.40:0/3681881551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56367afb3370 con 0x56367b1f91c0 2023-10-16T08:38:58.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa2097fa640 1 -- 172.21.15.40:0/3681881551 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa220015440 con 0x56367b1f91c0 2023-10-16T08:38:58.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa2097fa640 1 -- 172.21.15.40:0/3681881551 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa220015640 con 0x56367b1f91c0 2023-10-16T08:38:58.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 10 client.4659 fetch_fsmap learned FSMap version 6 2023-10-16T08:38:58.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 1 -- 172.21.15.40:0/3681881551 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x56367afc1460 con 0x56367b1f91c0 2023-10-16T08:38:58.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa2097fa640 1 -- 172.21.15.40:0/3681881551 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56367afb3370 con 0x56367b1f91c0 2023-10-16T08:38:58.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa2097fa640 1 -- 172.21.15.40:0/3681881551 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa2200157a0 con 0x56367b1f91c0 2023-10-16T08:38:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 10 client.4659 fetch_fsmap finished waiting for FSMap version 6 2023-10-16T08:38:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 10 client.4659 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T08:38:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 20 client.4659 populate_metadata read hostname 'smithi040' 2023-10-16T08:38:58.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 1 --2- 172.21.15.40:0/3681881551 >> [v2:172.21.15.40:6834/148507680,v1:172.21.15.40:6835/148507680] conn(0x56367b207930 0x56367b209d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 4 client.4659 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2023-10-16T08:38:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 1 -- 172.21.15.40:0/3681881551 --> [v2:172.21.15.40:6834/148507680,v1:172.21.15.40:6835/148507680] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56367af992d0 con 0x56367b207930 2023-10-16T08:38:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa22e4a1640 1 --2- 172.21.15.40:0/3681881551 >> [v2:172.21.15.40:6834/148507680,v1:172.21.15.40:6835/148507680] conn(0x56367b207930 0x56367b209d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa22e4a1640 1 --2- 172.21.15.40:0/3681881551 >> [v2:172.21.15.40:6834/148507680,v1:172.21.15.40:6835/148507680] conn(0x56367b207930 0x56367b209d20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=56b44573cfa7cf11 server_cookie=9daac3d0f592faaf in_seq=0 out_seq=0 2023-10-16T08:38:58.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa2097fa640 10 client.4659 ms_handle_connect on v2:172.21.15.40:6834/148507680 2023-10-16T08:38:58.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa2097fa640 1 -- 172.21.15.40:0/3681881551 <== mds.0 v2:172.21.15.40:6834/148507680 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18209 (crc 0 0 0) 0x56367af992d0 con 0x56367b207930 2023-10-16T08:38:58.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa2097fa640 10 client.4659 handle_command_reply: tid=0 2023-10-16T08:38:58.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.371+0000 7fa23384d000 1 client.4659 shutdown 2023-10-16T08:38:58.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.375+0000 7fa23384d000 1 -- 172.21.15.40:0/3681881551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f91c0 msgr2=0x56367b1ba2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:58.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.375+0000 7fa23384d000 1 --2- 172.21.15.40:0/3681881551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f91c0 0x56367b1ba2e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fa220007d10 tx=0x7fa220010590 comp rx=0 tx=0).stop 2023-10-16T08:38:58.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.375+0000 7fa23384d000 1 -- 172.21.15.40:0/3681881551 shutdown_connections 2023-10-16T08:38:58.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.375+0000 7fa23384d000 1 --2- 172.21.15.40:0/3681881551 >> [v2:172.21.15.40:6834/148507680,v1:172.21.15.40:6835/148507680] conn(0x56367b207930 0x56367b209d20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.375+0000 7fa23384d000 1 --2- 172.21.15.40:0/3681881551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f91c0 0x56367b1ba2e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.375+0000 7fa23384d000 1 --2- 172.21.15.40:0/3681881551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367b1f88a0 0x56367b1b9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.375+0000 7fa23384d000 1 --2- 172.21.15.40:0/3681881551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56367b1aec50 0x56367b1b9860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.375+0000 7fa2097fa640 0 client.4659 ms_handle_reset on v2:172.21.15.40:6834/148507680 2023-10-16T08:38:58.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.375+0000 7fa23384d000 1 -- 172.21.15.40:0/3681881551 >> 172.21.15.40:0/3681881551 conn(0x56367b1a5fc0 msgr2=0x56367b1ad6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:58.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.375+0000 7fa23384d000 1 -- 172.21.15.40:0/3681881551 shutdown_connections 2023-10-16T08:38:58.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.375+0000 7fa23384d000 1 -- 172.21.15.40:0/3681881551 wait complete. 2023-10-16T08:38:58.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.375+0000 7fa23384d000 20 client.4659 trim_cache size 0 max 16384 2023-10-16T08:38:58.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.387+0000 7fa23384d000 1 Processor -- start 2023-10-16T08:38:58.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.387+0000 7fa23384d000 1 -- start start 2023-10-16T08:38:58.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.387+0000 7fa23384d000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56367b1a5fc0 0x56367b252f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.387+0000 7fa23384d000 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1aec50 0x56367b253460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.387+0000 7fa23384d000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367b1f88a0 0x56367b2539a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.387+0000 7fa23384d000 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367acb9940 con 0x56367b1aec50 2023-10-16T08:38:58.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.387+0000 7fa23384d000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367b0048b0 con 0x56367b1f88a0 2023-10-16T08:38:58.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.387+0000 7fa23384d000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x56367b0022a0 con 0x56367b1a5fc0 2023-10-16T08:38:58.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.387+0000 7fa22dca0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1aec50 0x56367b253460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.387+0000 7fa22dca0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1aec50 0x56367b253460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34186/0 (socket says 172.21.15.40:34186) 2023-10-16T08:38:58.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.387+0000 7fa22dca0640 1 -- 172.21.15.40:0/629693390 learned_addr learned my addr 172.21.15.40:0/629693390 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:58.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa22dca0640 1 -- 172.21.15.40:0/629693390 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56367b1a5fc0 msgr2=0x56367b252f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:58.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa22dca0640 1 --2- 172.21.15.40:0/629693390 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56367b1a5fc0 0x56367b252f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa22dca0640 1 -- 172.21.15.40:0/629693390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367b1f88a0 msgr2=0x56367b2539a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:58.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa22dca0640 1 --2- 172.21.15.40:0/629693390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367b1f88a0 0x56367b2539a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa22dca0640 1 -- 172.21.15.40:0/629693390 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56367afa7760 con 0x56367b1aec50 2023-10-16T08:38:58.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa22dca0640 1 --2- 172.21.15.40:0/629693390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1aec50 0x56367b253460 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa224005ad0 tx=0x7fa224011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca942ecae2cac7f7 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:58.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa21cff9640 1 -- 172.21.15.40:0/629693390 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa224012410 con 0x56367b1aec50 2023-10-16T08:38:58.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa21cff9640 1 -- 172.21.15.40:0/629693390 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa22401a030 con 0x56367b1aec50 2023-10-16T08:38:58.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa21cff9640 1 -- 172.21.15.40:0/629693390 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2240127b0 con 0x56367b1aec50 2023-10-16T08:38:58.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 -- 172.21.15.40:0/629693390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1aec50 msgr2=0x56367b253460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:58.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 --2- 172.21.15.40:0/629693390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1aec50 0x56367b253460 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa224005ad0 tx=0x7fa224011000 comp rx=0 tx=0).stop 2023-10-16T08:38:58.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 -- 172.21.15.40:0/629693390 shutdown_connections 2023-10-16T08:38:58.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 --2- 172.21.15.40:0/629693390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367b1f88a0 0x56367b2539a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 --2- 172.21.15.40:0/629693390 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1aec50 0x56367b253460 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 --2- 172.21.15.40:0/629693390 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56367b1a5fc0 0x56367b252f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 -- 172.21.15.40:0/629693390 >> 172.21.15.40:0/629693390 conn(0x56367b1ba050 msgr2=0x56367b20ac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:58.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 -- 172.21.15.40:0/629693390 shutdown_connections 2023-10-16T08:38:58.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 -- 172.21.15.40:0/629693390 wait complete. 2023-10-16T08:38:58.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 -- start start 2023-10-16T08:38:58.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 Processor -- start 2023-10-16T08:38:58.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56367b1a5fc0 0x56367b1f73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367b1aec50 0x56367b2523d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f88a0 0x56367b252910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367afd5a10 con 0x56367b1f88a0 2023-10-16T08:38:58.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x56367afad6c0 con 0x56367b1aec50 2023-10-16T08:38:58.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x56367acf91b0 con 0x56367b1a5fc0 2023-10-16T08:38:58.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 10 client.0 _ll_register_callbacks cb 0x7fa2180eb620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-16T08:38:58.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa22eca2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f88a0 0x56367b252910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa22eca2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f88a0 0x56367b252910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34192/0 (socket says 172.21.15.40:34192) 2023-10-16T08:38:58.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa22eca2640 1 -- 172.21.15.40:0/2746587559 learned_addr learned my addr 172.21.15.40:0/2746587559 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:58.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa22eca2640 1 -- 172.21.15.40:0/2746587559 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56367b1a5fc0 msgr2=0x56367b1f73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:58.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa22eca2640 1 --2- 172.21.15.40:0/2746587559 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56367b1a5fc0 0x56367b1f73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa22eca2640 1 -- 172.21.15.40:0/2746587559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367b1aec50 msgr2=0x56367b2523d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:58.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa22eca2640 1 --2- 172.21.15.40:0/2746587559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367b1aec50 0x56367b2523d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa22eca2640 1 -- 172.21.15.40:0/2746587559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa22806b540 con 0x56367b1f88a0 2023-10-16T08:38:58.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa22eca2640 1 --2- 172.21.15.40:0/2746587559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f88a0 0x56367b252910 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa220000f90 tx=0x7fa220004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=703903b51d626cad server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:58.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa2097fa640 10 client.0 ms_handle_connect on v2:172.21.15.40:3300/0 2023-10-16T08:38:58.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa2097fa640 1 -- 172.21.15.40:0/2746587559 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa22000e050 con 0x56367b1f88a0 2023-10-16T08:38:58.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa2097fa640 1 -- 172.21.15.40:0/2746587559 <== mon.0 v2:172.21.15.40:3300/0 2 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa220007000 con 0x56367b1f88a0 2023-10-16T08:38:58.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 -- 172.21.15.40:0/2746587559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56367b1b2560 con 0x56367b1f88a0 2023-10-16T08:38:58.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa2097fa640 1 -- 172.21.15.40:0/2746587559 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_get_version_reply(handle=1 version=6) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa220007900 con 0x56367b1f88a0 2023-10-16T08:38:58.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 10 client.4665 fetch_fsmap learned FSMap version 6 2023-10-16T08:38:58.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 -- 172.21.15.40:0/2746587559 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,fsmap=6,monmap=2+}) v3 -- 0x56367acf8cc0 con 0x56367b1f88a0 2023-10-16T08:38:58.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa2097fa640 1 -- 172.21.15.40:0/2746587559 <== mon.0 v2:172.21.15.40:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56367b1b2560 con 0x56367b1f88a0 2023-10-16T08:38:58.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa2097fa640 1 -- 172.21.15.40:0/2746587559 <== mon.0 v2:172.21.15.40:3300/0 5 ==== fsmap(e 6) v1 ==== 3294+0+0 (secure 0 0 0) 0x7fa22002e020 con 0x56367b1f88a0 2023-10-16T08:38:58.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 10 client.4665 fetch_fsmap finished waiting for FSMap version 6 2023-10-16T08:38:58.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 10 client.4665 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-16T08:38:58.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 20 client.4665 populate_metadata read hostname 'smithi040' 2023-10-16T08:38:58.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 --2- 172.21.15.40:0/2746587559 >> [v2:172.21.15.40:6834/148507680,v1:172.21.15.40:6835/148507680] conn(0x56367b1f91c0 0x56367b2602a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 4 client.4665 mds_command: new command op to 4287 tid=0 multi_id=0 [{"prefix": "status"}] 2023-10-16T08:38:58.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa23384d000 1 -- 172.21.15.40:0/2746587559 --> [v2:172.21.15.40:6834/148507680,v1:172.21.15.40:6835/148507680] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56367af976f0 con 0x56367b1f91c0 2023-10-16T08:38:58.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.391+0000 7fa22e4a1640 1 --2- 172.21.15.40:0/2746587559 >> [v2:172.21.15.40:6834/148507680,v1:172.21.15.40:6835/148507680] conn(0x56367b1f91c0 0x56367b2602a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa2097fa640 10 client.4665 ms_handle_connect on v2:172.21.15.40:6834/148507680 2023-10-16T08:38:58.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa22e4a1640 1 --2- 172.21.15.40:0/2746587559 >> [v2:172.21.15.40:6834/148507680,v1:172.21.15.40:6835/148507680] conn(0x56367b1f91c0 0x56367b2602a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6fa121bdb915f91 server_cookie=c55c9706ee0f97f5 in_seq=0 out_seq=0 2023-10-16T08:38:58.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa2097fa640 1 -- 172.21.15.40:0/2746587559 <== mds.0 v2:172.21.15.40:6834/148507680 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+323 (crc 0 0 0) 0x56367af976f0 con 0x56367b1f91c0 2023-10-16T08:38:58.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa2097fa640 10 client.4665 handle_command_reply: tid=0 2023-10-16T08:38:58.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa23384d000 1 client.4665 shutdown 2023-10-16T08:38:58.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa23384d000 1 -- 172.21.15.40:0/2746587559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f88a0 msgr2=0x56367b252910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:58.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa23384d000 1 --2- 172.21.15.40:0/2746587559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f88a0 0x56367b252910 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fa220000f90 tx=0x7fa220004ac0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa23384d000 1 -- 172.21.15.40:0/2746587559 shutdown_connections 2023-10-16T08:38:58.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa23384d000 1 --2- 172.21.15.40:0/2746587559 >> [v2:172.21.15.40:6834/148507680,v1:172.21.15.40:6835/148507680] conn(0x56367b1f91c0 0x56367b2602a0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa23384d000 1 --2- 172.21.15.40:0/2746587559 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x56367b1f88a0 0x56367b252910 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa23384d000 1 --2- 172.21.15.40:0/2746587559 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56367b1aec50 0x56367b2523d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa23384d000 1 --2- 172.21.15.40:0/2746587559 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x56367b1a5fc0 0x56367b1f73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa2097fa640 0 client.4665 ms_handle_reset on v2:172.21.15.40:6834/148507680 2023-10-16T08:38:58.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa23384d000 1 -- 172.21.15.40:0/2746587559 >> 172.21.15.40:0/2746587559 conn(0x56367b1ba050 msgr2=0x56367b20b330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:58.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa23384d000 1 -- 172.21.15.40:0/2746587559 shutdown_connections 2023-10-16T08:38:58.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa23384d000 1 -- 172.21.15.40:0/2746587559 wait complete. 2023-10-16T08:38:58.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa23384d000 20 client.4665 trim_cache size 0 max 16384 2023-10-16T08:38:58.420 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-16T08:38:58.420 INFO:teuthology.orchestra.run.smithi040.stdout: "cluster_fsid": "cc6c59f7-0179-4f54-b71d-89fa67221543", 2023-10-16T08:38:58.421 INFO:teuthology.orchestra.run.smithi040.stdout: "whoami": 0, 2023-10-16T08:38:58.421 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4287, 2023-10-16T08:38:58.421 INFO:teuthology.orchestra.run.smithi040.stdout: "want_state": "up:active", 2023-10-16T08:38:58.421 INFO:teuthology.orchestra.run.smithi040.stdout: "state": "up:active", 2023-10-16T08:38:58.421 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs", 2023-10-16T08:38:58.422 INFO:teuthology.orchestra.run.smithi040.stdout: "rank_uptime": 12.46885522, 2023-10-16T08:38:58.422 INFO:teuthology.orchestra.run.smithi040.stdout: "mdsmap_epoch": 6, 2023-10-16T08:38:58.422 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch": 20, 2023-10-16T08:38:58.422 INFO:teuthology.orchestra.run.smithi040.stdout: "osdmap_epoch_barrier": 20, 2023-10-16T08:38:58.422 INFO:teuthology.orchestra.run.smithi040.stdout: "uptime": 19.808597625000001 2023-10-16T08:38:58.423 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-16T08:38:58.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa230705640 1 -- 172.21.15.40:0/2011686016 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa20c0664a0 msgr2=0x7fa20c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:58.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa230705640 1 --2- 172.21.15.40:0/2011686016 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa20c0664a0 0x7fa20c068960 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fa2281555f0 tx=0x7fa224009040 comp rx=0 tx=0).stop 2023-10-16T08:38:58.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa230705640 1 -- 172.21.15.40:0/2011686016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2280ade00 msgr2=0x7fa2280b1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:58.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa230705640 1 --2- 172.21.15.40:0/2011686016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2280ade00 0x7fa2280b1580 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fa21801cdb0 tx=0x7fa21801ad50 comp rx=0 tx=0).stop 2023-10-16T08:38:58.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa230705640 1 -- 172.21.15.40:0/2011686016 shutdown_connections 2023-10-16T08:38:58.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa230705640 1 --2- 172.21.15.40:0/2011686016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa22815d320 0x7fa2281f93f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa230705640 1 --2- 172.21.15.40:0/2011686016 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa22815cf40 0x7fa22814a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa230705640 1 --2- 172.21.15.40:0/2011686016 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa20c0664a0 0x7fa20c068960 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa230705640 1 --2- 172.21.15.40:0/2011686016 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa2280ade00 0x7fa2280b1580 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa230705640 1 -- 172.21.15.40:0/2011686016 >> 172.21.15.40:0/2011686016 conn(0x7fa2280ce960 msgr2=0x7fa2280c4980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:58.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa230705640 1 -- 172.21.15.40:0/2011686016 shutdown_connections 2023-10-16T08:38:58.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.395+0000 7fa230705640 1 -- 172.21.15.40:0/2011686016 wait complete. 2023-10-16T08:38:58.427 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-10-16T08:38:58.452 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-10-16T08:38:58.452 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-16T08:38:58.452 INFO:tasks.ceph_fuse:client.0 config is {} 2023-10-16T08:38:58.453 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T08:38:58.453 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T08:38:58.454 INFO:tasks.ceph_fuse:client.2 config is {} 2023-10-16T08:38:58.454 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T08:38:58.454 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-16T08:38:58.455 INFO:tasks.ceph_fuse:client.1 config is {} 2023-10-16T08:38:58.455 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T08:38:58.455 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-16T08:38:58.455 INFO:tasks.ceph_fuse:client.3 config is {} 2023-10-16T08:38:58.456 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-10-16T08:38:58.456 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-16T08:38:58.456 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:38:58.456 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-10-16T08:38:58.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:38:58.463 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2023-10-16T08:38:58.525 INFO:teuthology.orchestra.run.smithi040.stderr:Cannot find device "ceph-brx" 2023-10-16T08:38:58.526 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:38:58.527 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:38:58.527 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2023-10-16T08:38:58.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:38:58.535 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2023-10-16T08:38:58.594 INFO:teuthology.orchestra.run.smithi086.stderr:Cannot find device "ceph-brx" 2023-10-16T08:38:58.595 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:38:58.596 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-10-16T08:38:58.596 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T08:38:58.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.719+0000 7f40c42ce640 1 Processor -- start 2023-10-16T08:38:58.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.719+0000 7f40c42ce640 1 -- start start 2023-10-16T08:38:58.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c42ce640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f40bc0cae50 0x7f40bc06b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c42ce640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40bc0adca0 con 0x7f40bc15b040 2023-10-16T08:38:58.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c42ce640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40bc0c5840 con 0x7f40bc0adec0 2023-10-16T08:38:58.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c42ce640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40bc0cf890 con 0x7f40bc0cae50 2023-10-16T08:38:58.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c206a640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f40bc15b040 0x7f40bc0ca740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50430/0 (socket says 172.21.15.40:50430) 2023-10-16T08:38:58.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c206a640 1 -- 172.21.15.40:0/1967599700 learned_addr learned my addr 172.21.15.40:0/1967599700 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:38:58.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1869640 1 --2- 172.21.15.40:0/1967599700 >> v2:172.21.15.86:3300/0 conn(0x7f40bc0cae50 0x7f40bc06b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1068640 1 -- 172.21.15.40:0/1967599700 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1940492614 0 0) 0x7f40bc0adca0 con 0x7f40bc15b040 2023-10-16T08:38:58.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1068640 1 -- 172.21.15.40:0/1967599700 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40a8003530 con 0x7f40bc15b040 2023-10-16T08:38:58.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1068640 1 -- 172.21.15.40:0/1967599700 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1126536355 0 0) 0x7f40a8003530 con 0x7f40bc15b040 2023-10-16T08:38:58.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1068640 1 -- 172.21.15.40:0/1967599700 >> v2:172.21.15.86:3300/0 conn(0x7f40bc0cae50 msgr2=0x7f40bc06b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1068640 1 --2- 172.21.15.40:0/1967599700 >> v2:172.21.15.86:3300/0 conn(0x7f40bc0cae50 0x7f40bc06b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1068640 1 -- 172.21.15.40:0/1967599700 >> v1:172.21.15.86:6789/0 conn(0x7f40bc0adec0 legacy=0x7f40bc06bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1068640 1 -- 172.21.15.40:0/1967599700 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc0cf5b0 con 0x7f40bc15b040 2023-10-16T08:38:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1068640 1 -- 172.21.15.40:0/1967599700 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f40b8001a00 con 0x7f40bc15b040 2023-10-16T08:38:58.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1068640 1 -- 172.21.15.40:0/1967599700 >> v1:172.21.15.40:6789/0 conn(0x7f40bc15b040 legacy=0x7f40bc0ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:58.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1068640 1 --2- 172.21.15.40:0/1967599700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40a80039f0 0x7f40a8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1068640 1 --2- 172.21.15.40:0/1967599700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a80047d0 0x7f40a8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1068640 1 --2- 172.21.15.40:0/1967599700 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40a8005290 0x7f40a800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1068640 1 -- 172.21.15.40:0/1967599700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc0cf890 con 0x7f40a80047d0 2023-10-16T08:38:58.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1068640 1 -- 172.21.15.40:0/1967599700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a800c1d0 con 0x7f40a80039f0 2023-10-16T08:38:58.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1068640 1 -- 172.21.15.40:0/1967599700 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a800c390 con 0x7f40a8005290 2023-10-16T08:38:58.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1869640 1 --2- 172.21.15.40:0/1967599700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a80047d0 0x7f40a8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c206a640 1 --2- 172.21.15.40:0/1967599700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40a80039f0 0x7f40a8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1869640 1 -- 172.21.15.40:0/1967599700 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40a8005290 msgr2=0x7f40a800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:58.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c286b640 1 --2- 172.21.15.40:0/1967599700 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40a8005290 0x7f40a800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1869640 1 --2- 172.21.15.40:0/1967599700 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40a8005290 0x7f40a800b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1869640 1 -- 172.21.15.40:0/1967599700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40a80039f0 msgr2=0x7f40a8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:38:58.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1869640 1 --2- 172.21.15.40:0/1967599700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40a80039f0 0x7f40a8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c1869640 1 -- 172.21.15.40:0/1967599700 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a8003530 con 0x7f40a80047d0 2023-10-16T08:38:58.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.723+0000 7f40c286b640 1 -- 172.21.15.40:0/1967599700 reap_dead start 2023-10-16T08:38:58.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c1869640 1 --2- 172.21.15.40:0/1967599700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a80047d0 0x7f40a8004c50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009ae0 tx=0x7f40ac01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42d8951726e728d0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:58.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c1068640 1 -- 172.21.15.40:0/1967599700 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac0a6510 con 0x7f40a80047d0 2023-10-16T08:38:58.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c1068640 1 -- 172.21.15.40:0/1967599700 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40ac0a6670 con 0x7f40a80047d0 2023-10-16T08:38:58.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c1068640 1 -- 172.21.15.40:0/1967599700 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40ac0a6a20 con 0x7f40a80047d0 2023-10-16T08:38:58.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c42ce640 1 -- 172.21.15.40:0/1967599700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a80047d0 msgr2=0x7f40a8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:58.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c42ce640 1 --2- 172.21.15.40:0/1967599700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a80047d0 0x7f40a8004c50 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f40ac009ae0 tx=0x7f40ac01ca20 comp rx=0 tx=0).stop 2023-10-16T08:38:58.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c42ce640 1 -- 172.21.15.40:0/1967599700 shutdown_connections 2023-10-16T08:38:58.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c42ce640 1 --2- 172.21.15.40:0/1967599700 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a80047d0 0x7f40a8004c50 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c42ce640 1 -- 172.21.15.40:0/1967599700 >> 172.21.15.40:0/1967599700 conn(0x7f40bc0ce960 msgr2=0x7f40bc0ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:58.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c42ce640 1 -- 172.21.15.40:0/1967599700 shutdown_connections 2023-10-16T08:38:58.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c42ce640 1 -- 172.21.15.40:0/1967599700 wait complete. 2023-10-16T08:38:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c42ce640 1 Processor -- start 2023-10-16T08:38:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c42ce640 1 -- start start 2023-10-16T08:38:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c42ce640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a80047d0 0x7f40bc1f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c42ce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40bc0c5a30 0x7f40bc0c2da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c42ce640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40bc15dc60 0x7f40bc0c32e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c42ce640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc0c3d10 con 0x7f40a80047d0 2023-10-16T08:38:58.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c42ce640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc15c060 con 0x7f40bc0c5a30 2023-10-16T08:38:58.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c42ce640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40bc0c3820 con 0x7f40bc15dc60 2023-10-16T08:38:58.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c206a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a80047d0 0x7f40bc1f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c206a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a80047d0 0x7f40bc1f50a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34212/0 (socket says 172.21.15.40:34212) 2023-10-16T08:38:58.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c206a640 1 -- 172.21.15.40:0/4204111940 learned_addr learned my addr 172.21.15.40:0/4204111940 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:38:58.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c286b640 1 --2- 172.21.15.40:0/4204111940 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40bc15dc60 0x7f40bc0c32e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c1869640 1 --2- 172.21.15.40:0/4204111940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40bc0c5a30 0x7f40bc0c2da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c206a640 1 -- 172.21.15.40:0/4204111940 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40bc15dc60 msgr2=0x7f40bc0c32e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:58.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c206a640 1 --2- 172.21.15.40:0/4204111940 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40bc15dc60 0x7f40bc0c32e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c206a640 1 -- 172.21.15.40:0/4204111940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40bc0c5a30 msgr2=0x7f40bc0c2da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:58.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c206a640 1 --2- 172.21.15.40:0/4204111940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40bc0c5a30 0x7f40bc0c2da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c206a640 1 -- 172.21.15.40:0/4204111940 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40bc0c3a70 con 0x7f40a80047d0 2023-10-16T08:38:58.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.727+0000 7f40c206a640 1 --2- 172.21.15.40:0/4204111940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a80047d0 0x7f40bc1f50a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f40b8007a40 tx=0x7f40b800e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:58.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.731+0000 7f40b2ffd640 1 -- 172.21.15.40:0/4204111940 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8001440 con 0x7f40a80047d0 2023-10-16T08:38:58.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.731+0000 7f40b2ffd640 1 -- 172.21.15.40:0/4204111940 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40b800edf0 con 0x7f40a80047d0 2023-10-16T08:38:58.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.731+0000 7f40b2ffd640 1 -- 172.21.15.40:0/4204111940 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40b8013640 con 0x7f40a80047d0 2023-10-16T08:38:58.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.731+0000 7f40c42ce640 1 -- 172.21.15.40:0/4204111940 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40bc0b3bd0 con 0x7f40a80047d0 2023-10-16T08:38:58.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.731+0000 7f40c42ce640 1 -- 172.21.15.40:0/4204111940 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40bc0b38f0 con 0x7f40a80047d0 2023-10-16T08:38:58.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.731+0000 7f40c42ce640 1 -- 172.21.15.40:0/4204111940 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40bc0b3750 con 0x7f40a80047d0 2023-10-16T08:38:58.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.731+0000 7f40b2ffd640 1 -- 172.21.15.40:0/4204111940 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f40b80137a0 con 0x7f40a80047d0 2023-10-16T08:38:58.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.731+0000 7f40b2ffd640 1 --2- 172.21.15.40:0/4204111940 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f40a00664d0 0x7f40a0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:38:58.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.731+0000 7f40b2ffd640 1 -- 172.21.15.40:0/4204111940 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f40b8084300 con 0x7f40a80047d0 2023-10-16T08:38:58.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.731+0000 7f40c1869640 1 --2- 172.21.15.40:0/4204111940 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f40a00664d0 0x7f40a0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:38:58.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.731+0000 7f40c1869640 1 --2- 172.21.15.40:0/4204111940 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f40a00664d0 0x7f40a0068990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0af000 tx=0x7f40ac008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:38:58.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.735+0000 7f40b2ffd640 1 -- 172.21.15.40:0/4204111940 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f40bc0b3750 con 0x7f40a80047d0 2023-10-16T08:38:58.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.923+0000 7f40c42ce640 1 -- 172.21.15.40:0/4204111940 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f40bc0b3750 con 0x7f40a80047d0 2023-10-16T08:38:58.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.923+0000 7f40b2ffd640 1 -- 172.21.15.40:0/4204111940 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f40bc0b3750 con 0x7f40a80047d0 2023-10-16T08:38:58.928 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T08:38:58.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.923+0000 7f40c42ce640 1 -- 172.21.15.40:0/4204111940 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f40a00664d0 msgr2=0x7f40a0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:58.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.923+0000 7f40c42ce640 1 --2- 172.21.15.40:0/4204111940 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f40a00664d0 0x7f40a0068990 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f40ac0af000 tx=0x7f40ac008040 comp rx=0 tx=0).stop 2023-10-16T08:38:58.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.923+0000 7f40c42ce640 1 -- 172.21.15.40:0/4204111940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a80047d0 msgr2=0x7f40bc1f50a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:38:58.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.923+0000 7f40c42ce640 1 --2- 172.21.15.40:0/4204111940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a80047d0 0x7f40bc1f50a0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f40b8007a40 tx=0x7f40b800e530 comp rx=0 tx=0).stop 2023-10-16T08:38:58.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.923+0000 7f40c42ce640 1 -- 172.21.15.40:0/4204111940 shutdown_connections 2023-10-16T08:38:58.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.923+0000 7f40c42ce640 1 --2- 172.21.15.40:0/4204111940 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40bc15dc60 0x7f40bc0c32e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.923+0000 7f40c42ce640 1 --2- 172.21.15.40:0/4204111940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40bc0c5a30 0x7f40bc0c2da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.923+0000 7f40c42ce640 1 --2- 172.21.15.40:0/4204111940 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f40a00664d0 0x7f40a0068990 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.923+0000 7f40c42ce640 1 --2- 172.21.15.40:0/4204111940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a80047d0 0x7f40bc1f50a0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:38:58.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.923+0000 7f40c42ce640 1 -- 172.21.15.40:0/4204111940 >> 172.21.15.40:0/4204111940 conn(0x7f40bc0ce960 msgr2=0x7f40bc0ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:38:58.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.923+0000 7f40c42ce640 1 -- 172.21.15.40:0/4204111940 shutdown_connections 2023-10-16T08:38:58.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:38:58.923+0000 7f40c42ce640 1 -- 172.21.15.40:0/4204111940 wait complete. 2023-10-16T08:38:58.946 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T08:38:58.947 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T08:38:58.947 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-10-16T08:38:58.947 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-16T08:38:58.947 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-16T08:38:58.947 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T08:38:58.948 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T08:38:58.948 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T08:38:58.948 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:38:58.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:38:58.949 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-10-16T08:38:58.954 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:38:58.954 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:38:58.955 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:38:58.955 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:38:58.955 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-10-16T08:38:58.955 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:38:58.956 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:38:58.956 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:38:58.956 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2023-10-16T08:38:58.956 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:38:58.956 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:38:58.957 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2023-10-16T08:38:58.957 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T08:38:58.957 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-10-16T08:38:58.957 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2023-10-16T08:38:58.957 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-16T08:38:58.957 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42487sec preferred_lft 42487sec 2023-10-16T08:38:58.958 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link 2023-10-16T08:38:58.958 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:38:58.958 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:38:58.958 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-10-16T08:38:58.958 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2023-10-16T08:38:58.959 INFO:teuthology.orchestra.run.smithi040.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T08:38:58.959 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 02:42:65:e1:22:0d brd ff:ff:ff:ff:ff:ff 2023-10-16T08:38:58.959 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T08:38:58.959 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:38:58.960 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-16T08:38:58.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:38:58.960 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-16T08:38:58.961 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-16T08:38:58.961 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add name ceph-brx type bridge 2023-10-16T08:38:58.961 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr flush dev ceph-brx 2023-10-16T08:38:58.962 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set ceph-brx up 2023-10-16T08:38:58.962 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-16T08:38:58.962 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-16T08:38:59.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:38:59.033 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T08:38:59.086 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-10-16T08:38:59.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:38:59.088 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-10-16T08:38:59.148 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-10-16T08:38:59.148 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T08:38:59.148 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-16T08:38:59.148 INFO:teuthology.orchestra.run.smithi040.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T08:38:59.149 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-16T08:38:59.149 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T08:38:59.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:38:59.150 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-16T08:38:59.150 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-16T08:38:59.150 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-16T08:38:59.150 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-16T08:38:59.151 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-16T08:38:59.151 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-16T08:38:59.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:38:59.228 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-10-16T08:38:59.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:38:59.275 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2023-10-16T08:38:59.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:38:59.323 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-16T08:38:59.324 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-16T08:38:59.324 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:38:59.324 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-16T08:38:59.325 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-16T08:38:59.395 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-16T08:38:59.396 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:38:59.396 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-16T08:38:59.396 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-16T08:38:59.396 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-16T08:38:59.397 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-16T08:38:59.397 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-16T08:38:59.397 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-16T08:38:59.398 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-16T08:38:59.398 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-16T08:38:59.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:38:59.515 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-16T08:38:59.515 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-16T08:38:59.516 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.0 up 2023-10-16T08:38:59.516 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.0 master ceph-brx 2023-10-16T08:38:59.516 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-16T08:38:59.536 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-16T08:38:59.536 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:38:59.537 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-16T08:38:59.583 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-16T08:38:59.583 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:38:59.583 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-16T08:38:59.631 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2023-10-16T08:38:59.687 DEBUG:teuthology.orchestra.run.smithi040:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:38:59.737 INFO:teuthology.orchestra.run.smithi040.stdout:/sys 2023-10-16T08:38:59.737 INFO:teuthology.orchestra.run.smithi040.stdout:/proc 2023-10-16T08:38:59.738 INFO:teuthology.orchestra.run.smithi040.stdout:/dev 2023-10-16T08:38:59.738 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/pts 2023-10-16T08:38:59.738 INFO:teuthology.orchestra.run.smithi040.stdout:/run 2023-10-16T08:38:59.738 INFO:teuthology.orchestra.run.smithi040.stdout:/ 2023-10-16T08:38:59.739 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/security 2023-10-16T08:38:59.739 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/shm 2023-10-16T08:38:59.739 INFO:teuthology.orchestra.run.smithi040.stdout:/run/lock 2023-10-16T08:38:59.739 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/cgroup 2023-10-16T08:38:59.739 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/pstore 2023-10-16T08:38:59.739 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/bpf 2023-10-16T08:38:59.740 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:38:59.740 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/hugepages 2023-10-16T08:38:59.740 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/mqueue 2023-10-16T08:38:59.740 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/debug 2023-10-16T08:38:59.740 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/tracing 2023-10-16T08:38:59.741 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/fs/nfsd 2023-10-16T08:38:59.741 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/fuse/connections 2023-10-16T08:38:59.741 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/config 2023-10-16T08:38:59.741 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:38:59.741 INFO:teuthology.orchestra.run.smithi040.stdout:/run/rpc_pipefs 2023-10-16T08:38:59.742 INFO:teuthology.orchestra.run.smithi040.stdout:/run/user/1000 2023-10-16T08:38:59.742 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph 2023-10-16T08:38:59.742 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:38:59.742 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:38:59.742 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:38:59.743 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:38:59.743 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns 2023-10-16T08:38:59.743 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:38:59.743 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:38:59.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:38:59.744 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2023-10-16T08:38:59.795 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T08:38:59.796 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-16T08:38:59.840 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2023-10-16T08:38:59.853 DEBUG:teuthology.orchestra.run.smithi040:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:38:59.900 INFO:teuthology.orchestra.run.smithi040.stdout:/sys 2023-10-16T08:38:59.900 INFO:teuthology.orchestra.run.smithi040.stdout:/proc 2023-10-16T08:38:59.900 INFO:teuthology.orchestra.run.smithi040.stdout:/dev 2023-10-16T08:38:59.900 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/pts 2023-10-16T08:38:59.901 INFO:teuthology.orchestra.run.smithi040.stdout:/run 2023-10-16T08:38:59.901 INFO:teuthology.orchestra.run.smithi040.stdout:/ 2023-10-16T08:38:59.901 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/security 2023-10-16T08:38:59.901 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/shm 2023-10-16T08:38:59.902 INFO:teuthology.orchestra.run.smithi040.stdout:/run/lock 2023-10-16T08:38:59.902 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/cgroup 2023-10-16T08:38:59.902 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/pstore 2023-10-16T08:38:59.902 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/bpf 2023-10-16T08:38:59.902 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:38:59.903 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/hugepages 2023-10-16T08:38:59.903 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/mqueue 2023-10-16T08:38:59.903 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/debug 2023-10-16T08:38:59.903 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/tracing 2023-10-16T08:38:59.904 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/fs/nfsd 2023-10-16T08:38:59.904 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/fuse/connections 2023-10-16T08:38:59.904 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/config 2023-10-16T08:38:59.904 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:38:59.904 INFO:teuthology.orchestra.run.smithi040.stdout:/run/rpc_pipefs 2023-10-16T08:38:59.905 INFO:teuthology.orchestra.run.smithi040.stdout:/run/user/1000 2023-10-16T08:38:59.905 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph 2023-10-16T08:38:59.905 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:38:59.905 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:38:59.905 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:38:59.906 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:38:59.906 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns 2023-10-16T08:38:59.906 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:38:59.907 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:38:59.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:38:59.909 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2023-10-16T08:38:59.926 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:2023-10-16T08:38:59.919+0000 7f78fd039580 -1 init, newargv = 0x55df3f690eb0 newargc=15 2023-10-16T08:38:59.926 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:ceph-fuse[45077]: starting ceph client 2023-10-16T08:38:59.933 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:ceph-fuse[45077]: starting fuse 2023-10-16T08:39:01.031 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2023-10-16T08:39:01.048 DEBUG:teuthology.orchestra.run.smithi040:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:39:01.102 INFO:teuthology.orchestra.run.smithi040.stdout:/sys 2023-10-16T08:39:01.102 INFO:teuthology.orchestra.run.smithi040.stdout:/proc 2023-10-16T08:39:01.102 INFO:teuthology.orchestra.run.smithi040.stdout:/dev 2023-10-16T08:39:01.102 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/pts 2023-10-16T08:39:01.103 INFO:teuthology.orchestra.run.smithi040.stdout:/run 2023-10-16T08:39:01.103 INFO:teuthology.orchestra.run.smithi040.stdout:/ 2023-10-16T08:39:01.103 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/security 2023-10-16T08:39:01.103 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/shm 2023-10-16T08:39:01.103 INFO:teuthology.orchestra.run.smithi040.stdout:/run/lock 2023-10-16T08:39:01.104 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/cgroup 2023-10-16T08:39:01.104 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/pstore 2023-10-16T08:39:01.104 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/bpf 2023-10-16T08:39:01.104 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:39:01.104 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/hugepages 2023-10-16T08:39:01.105 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/mqueue 2023-10-16T08:39:01.105 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/debug 2023-10-16T08:39:01.105 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/tracing 2023-10-16T08:39:01.105 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/fs/nfsd 2023-10-16T08:39:01.105 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/fuse/connections 2023-10-16T08:39:01.106 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/config 2023-10-16T08:39:01.106 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:39:01.106 INFO:teuthology.orchestra.run.smithi040.stdout:/run/rpc_pipefs 2023-10-16T08:39:01.106 INFO:teuthology.orchestra.run.smithi040.stdout:/run/user/1000 2023-10-16T08:39:01.106 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph 2023-10-16T08:39:01.107 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:39:01.107 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:39:01.107 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:39:01.107 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:39:01.107 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns 2023-10-16T08:39:01.107 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:39:01.108 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:39:01.108 INFO:teuthology.orchestra.run.smithi040.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-16T08:39:01.109 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:01.109 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2023-10-16T08:39:01.155 INFO:teuthology.orchestra.run.smithi040.stdout:45 2023-10-16T08:39:01.156 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-10-16T08:39:01.156 DEBUG:teuthology.orchestra.run.smithi040:> sudo stdin-killer -- python3 -c ' 2023-10-16T08:39:01.157 DEBUG:teuthology.orchestra.run.smithi040:> import glob 2023-10-16T08:39:01.157 DEBUG:teuthology.orchestra.run.smithi040:> import re 2023-10-16T08:39:01.157 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-10-16T08:39:01.158 DEBUG:teuthology.orchestra.run.smithi040:> import subprocess 2023-10-16T08:39:01.158 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-16T08:39:01.159 DEBUG:teuthology.orchestra.run.smithi040:> def _find_admin_socket(client_name): 2023-10-16T08:39:01.159 DEBUG:teuthology.orchestra.run.smithi040:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-16T08:39:01.159 DEBUG:teuthology.orchestra.run.smithi040:> files = glob.glob(asok_path) 2023-10-16T08:39:01.160 DEBUG:teuthology.orchestra.run.smithi040:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-16T08:39:01.160 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-16T08:39:01.160 DEBUG:teuthology.orchestra.run.smithi040:> # Given a non-glob path, it better be there 2023-10-16T08:39:01.160 DEBUG:teuthology.orchestra.run.smithi040:> if "*" not in asok_path: 2023-10-16T08:39:01.161 DEBUG:teuthology.orchestra.run.smithi040:> assert(len(files) == 1) 2023-10-16T08:39:01.161 DEBUG:teuthology.orchestra.run.smithi040:> return files[0] 2023-10-16T08:39:01.161 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-16T08:39:01.162 DEBUG:teuthology.orchestra.run.smithi040:> for f in files: 2023-10-16T08:39:01.162 DEBUG:teuthology.orchestra.run.smithi040:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T08:39:01.162 DEBUG:teuthology.orchestra.run.smithi040:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T08:39:01.163 DEBUG:teuthology.orchestra.run.smithi040:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T08:39:01.163 DEBUG:teuthology.orchestra.run.smithi040:> contents = proc_f.read() 2023-10-16T08:39:01.163 DEBUG:teuthology.orchestra.run.smithi040:> if mountpoint in contents: 2023-10-16T08:39:01.163 DEBUG:teuthology.orchestra.run.smithi040:> return f 2023-10-16T08:39:01.164 DEBUG:teuthology.orchestra.run.smithi040:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T08:39:01.164 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-16T08:39:01.164 DEBUG:teuthology.orchestra.run.smithi040:> print(_find_admin_socket("client.0")) 2023-10-16T08:39:01.165 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-10-16T08:39:01.284 INFO:teuthology.orchestra.run.smithi040.stdout:/var/run/ceph/ceph-client.0.45077.asok 2023-10-16T08:39:01.288 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T08:39:01.297 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.45077.asok 2023-10-16T08:39:01.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:01.297 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.45077.asok status 2023-10-16T08:39:01.449 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-16T08:39:01.450 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": { 2023-10-16T08:39:01.450 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_sha1": "c7471068426370a3754c2250e7729280233d552b", 2023-10-16T08:39:01.450 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_version": "ceph version 18.0.0-6675-gc7471068 (c7471068426370a3754c2250e7729280233d552b) reef (dev)", 2023-10-16T08:39:01.450 INFO:teuthology.orchestra.run.smithi040.stdout: "entity_id": "0", 2023-10-16T08:39:01.451 INFO:teuthology.orchestra.run.smithi040.stdout: "hostname": "smithi040", 2023-10-16T08:39:01.451 INFO:teuthology.orchestra.run.smithi040.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-16T08:39:01.451 INFO:teuthology.orchestra.run.smithi040.stdout: "pid": "45077", 2023-10-16T08:39:01.451 INFO:teuthology.orchestra.run.smithi040.stdout: "root": "/" 2023-10-16T08:39:01.451 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-16T08:39:01.452 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_count": 0, 2023-10-16T08:39:01.452 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_pinned_count": 0, 2023-10-16T08:39:01.452 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4680, 2023-10-16T08:39:01.452 INFO:teuthology.orchestra.run.smithi040.stdout: "inst": { 2023-10-16T08:39:01.452 INFO:teuthology.orchestra.run.smithi040.stdout: "name": { 2023-10-16T08:39:01.452 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2023-10-16T08:39:01.453 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4680 2023-10-16T08:39:01.453 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-16T08:39:01.453 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2023-10-16T08:39:01.453 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-16T08:39:01.453 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "192.168.0.1:0", 2023-10-16T08:39:01.453 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2597517428 2023-10-16T08:39:01.454 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-16T08:39:01.454 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-16T08:39:01.454 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2023-10-16T08:39:01.454 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-16T08:39:01.454 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "192.168.0.1:0", 2023-10-16T08:39:01.455 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 2597517428 2023-10-16T08:39:01.455 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-16T08:39:01.455 INFO:teuthology.orchestra.run.smithi040.stdout: "inst_str": "client.4680 192.168.0.1:0/2597517428", 2023-10-16T08:39:01.455 INFO:teuthology.orchestra.run.smithi040.stdout: "addr_str": "192.168.0.1:0/2597517428", 2023-10-16T08:39:01.455 INFO:teuthology.orchestra.run.smithi040.stdout: "inode_count": 1, 2023-10-16T08:39:01.455 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_epoch": 6, 2023-10-16T08:39:01.456 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch": 20, 2023-10-16T08:39:01.456 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch_barrier": 0, 2023-10-16T08:39:01.456 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklisted": false, 2023-10-16T08:39:01.456 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs" 2023-10-16T08:39:01.456 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-16T08:39:01.465 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T08:39:01.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.583+0000 7fdc63512640 1 Processor -- start 2023-10-16T08:39:01.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.587+0000 7fdc63512640 1 -- start start 2023-10-16T08:39:01.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.587+0000 7fdc63512640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc5c15b040 0x7fdc5c0ca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:01.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.587+0000 7fdc63512640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdc5c06c7c0 0x7fdc5c0cac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:01.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.587+0000 7fdc63512640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc5c0adca0 con 0x7fdc5c0adec0 2023-10-16T08:39:01.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.587+0000 7fdc63512640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc5c0c5840 con 0x7fdc5c06c7c0 2023-10-16T08:39:01.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.587+0000 7fdc63512640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c0cf890 con 0x7fdc5c15b040 2023-10-16T08:39:01.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.587+0000 7fdc61aaf640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdc5c06c7c0 0x7fdc5c0cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:01.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.587+0000 7fdc60aad640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fdc5c0adec0 0x7fdc5c0c8fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.40:36088/0 (socket says 172.21.15.40:36088) 2023-10-16T08:39:01.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.587+0000 7fdc60aad640 1 -- 172.21.15.40:0/3546956574 learned_addr learned my addr 172.21.15.40:0/3546956574 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:01.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.587+0000 7fdc612ae640 1 --2- 172.21.15.40:0/3546956574 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc5c15b040 0x7fdc5c0ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:01.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.587+0000 7fdc612ae640 1 -- 172.21.15.40:0/3546956574 >> v2:172.21.15.86:3300/0 conn(0x7fdc5c06c7c0 msgr2=0x7fdc5c0cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:01.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.587+0000 7fdc612ae640 1 --2- 172.21.15.40:0/3546956574 >> v2:172.21.15.86:3300/0 conn(0x7fdc5c06c7c0 0x7fdc5c0cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:01.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.587+0000 7fdc612ae640 1 -- 172.21.15.40:0/3546956574 >> v1:172.21.15.86:6789/0 conn(0x7fdc5c0adec0 legacy=0x7fdc5c0c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:01.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.587+0000 7fdc612ae640 1 -- 172.21.15.40:0/3546956574 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc5c0cf5b0 con 0x7fdc5c15b040 2023-10-16T08:39:01.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.587+0000 7fdc61aaf640 1 --2- 172.21.15.40:0/3546956574 >> v2:172.21.15.86:3300/0 conn(0x7fdc5c06c7c0 0x7fdc5c0cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:39:01.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.591+0000 7fdc612ae640 1 --2- 172.21.15.40:0/3546956574 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc5c15b040 0x7fdc5c0ca740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdc500097c0 tx=0x7fdc5001c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9da78b99cee095c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:01.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.591+0000 7fdc4ffff640 1 -- 172.21.15.40:0/3546956574 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500ab600 con 0x7fdc5c15b040 2023-10-16T08:39:01.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.591+0000 7fdc4ffff640 1 -- 172.21.15.40:0/3546956574 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc500a7050 con 0x7fdc5c15b040 2023-10-16T08:39:01.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.591+0000 7fdc4ffff640 1 -- 172.21.15.40:0/3546956574 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500ab980 con 0x7fdc5c15b040 2023-10-16T08:39:01.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.591+0000 7fdc63512640 1 -- 172.21.15.40:0/3546956574 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc5c15b040 msgr2=0x7fdc5c0ca740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:01.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.591+0000 7fdc63512640 1 --2- 172.21.15.40:0/3546956574 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc5c15b040 0x7fdc5c0ca740 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdc500097c0 tx=0x7fdc5001c330 comp rx=0 tx=0).stop 2023-10-16T08:39:01.597 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.591+0000 7fdc63512640 1 -- 172.21.15.40:0/3546956574 shutdown_connections 2023-10-16T08:39:01.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.591+0000 7fdc63512640 1 --2- 172.21.15.40:0/3546956574 >> v2:172.21.15.86:3300/0 conn(0x7fdc5c06c7c0 0x7fdc5c0cac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:01.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.591+0000 7fdc63512640 1 --2- 172.21.15.40:0/3546956574 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc5c15b040 0x7fdc5c0ca740 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:01.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.591+0000 7fdc63512640 1 -- 172.21.15.40:0/3546956574 >> 172.21.15.40:0/3546956574 conn(0x7fdc5c0ce960 msgr2=0x7fdc5c0ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:01.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.591+0000 7fdc63512640 1 -- 172.21.15.40:0/3546956574 shutdown_connections 2023-10-16T08:39:01.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.591+0000 7fdc63512640 1 -- 172.21.15.40:0/3546956574 wait complete. 2023-10-16T08:39:01.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.591+0000 7fdc63512640 1 Processor -- start 2023-10-16T08:39:01.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.591+0000 7fdc63512640 1 -- start start 2023-10-16T08:39:01.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc63512640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc5c06c7c0 0x7fdc5c155680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:01.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc63512640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc5c0adec0 0x7fdc5c0b1580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:01.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc63512640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5c15b040 0x7fdc5c154e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:01.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc63512640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c0c74f0 con 0x7fdc5c15b040 2023-10-16T08:39:01.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc63512640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c155f80 con 0x7fdc5c06c7c0 2023-10-16T08:39:01.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc63512640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5c0c71a0 con 0x7fdc5c0adec0 2023-10-16T08:39:01.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc61aaf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5c15b040 0x7fdc5c154e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:01.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc61aaf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5c15b040 0x7fdc5c154e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34214/0 (socket says 172.21.15.40:34214) 2023-10-16T08:39:01.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc61aaf640 1 -- 172.21.15.40:0/3052737576 learned_addr learned my addr 172.21.15.40:0/3052737576 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:01.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc612ae640 1 --2- 172.21.15.40:0/3052737576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc5c06c7c0 0x7fdc5c155680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:01.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc60aad640 1 --2- 172.21.15.40:0/3052737576 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc5c0adec0 0x7fdc5c0b1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:01.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc61aaf640 1 -- 172.21.15.40:0/3052737576 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc5c0adec0 msgr2=0x7fdc5c0b1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:01.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc61aaf640 1 --2- 172.21.15.40:0/3052737576 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc5c0adec0 0x7fdc5c0b1580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:01.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc61aaf640 1 -- 172.21.15.40:0/3052737576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc5c06c7c0 msgr2=0x7fdc5c155680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:01.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc61aaf640 1 --2- 172.21.15.40:0/3052737576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc5c06c7c0 0x7fdc5c155680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:01.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc61aaf640 1 -- 172.21.15.40:0/3052737576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc54009420 con 0x7fdc5c15b040 2023-10-16T08:39:01.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc61aaf640 1 --2- 172.21.15.40:0/3052737576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5c15b040 0x7fdc5c154e30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdc5400e7c0 tx=0x7fdc5400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:01.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc4dffb640 1 -- 172.21.15.40:0/3052737576 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc540095b0 con 0x7fdc5c15b040 2023-10-16T08:39:01.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc4dffb640 1 -- 172.21.15.40:0/3052737576 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5400c940 con 0x7fdc5c15b040 2023-10-16T08:39:01.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc63512640 1 -- 172.21.15.40:0/3052737576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc50009420 con 0x7fdc5c15b040 2023-10-16T08:39:01.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc4dffb640 1 -- 172.21.15.40:0/3052737576 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5400ccf0 con 0x7fdc5c15b040 2023-10-16T08:39:01.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc63512640 1 -- 172.21.15.40:0/3052737576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5c0b1f40 con 0x7fdc5c15b040 2023-10-16T08:39:01.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc63512640 1 -- 172.21.15.40:0/3052737576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc5c0c7630 con 0x7fdc5c15b040 2023-10-16T08:39:01.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc4dffb640 1 -- 172.21.15.40:0/3052737576 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdc54013470 con 0x7fdc5c15b040 2023-10-16T08:39:01.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc4dffb640 1 --2- 172.21.15.40:0/3052737576 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdc400664a0 0x7fdc40068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:01.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc4dffb640 1 -- 172.21.15.40:0/3052737576 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdc54083fc0 con 0x7fdc5c15b040 2023-10-16T08:39:01.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.595+0000 7fdc612ae640 1 --2- 172.21.15.40:0/3052737576 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdc400664a0 0x7fdc40068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:01.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.599+0000 7fdc612ae640 1 --2- 172.21.15.40:0/3052737576 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdc400664a0 0x7fdc40068960 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdc500ad000 tx=0x7fdc500a6000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:01.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.599+0000 7fdc4dffb640 1 -- 172.21.15.40:0/3052737576 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fdc540b53d0 con 0x7fdc5c15b040 2023-10-16T08:39:01.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.799+0000 7fdc63512640 1 -- 172.21.15.40:0/3052737576 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fdc5c04e9c0 con 0x7fdc5c15b040 2023-10-16T08:39:01.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.799+0000 7fdc4dffb640 1 -- 172.21.15.40:0/3052737576 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7fdc5c0c7630 con 0x7fdc5c15b040 2023-10-16T08:39:01.806 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T08:39:01.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.803+0000 7fdc63512640 1 -- 172.21.15.40:0/3052737576 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdc400664a0 msgr2=0x7fdc40068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:01.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.803+0000 7fdc63512640 1 --2- 172.21.15.40:0/3052737576 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdc400664a0 0x7fdc40068960 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fdc500ad000 tx=0x7fdc500a6000 comp rx=0 tx=0).stop 2023-10-16T08:39:01.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.803+0000 7fdc63512640 1 -- 172.21.15.40:0/3052737576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5c15b040 msgr2=0x7fdc5c154e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:01.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.803+0000 7fdc63512640 1 --2- 172.21.15.40:0/3052737576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5c15b040 0x7fdc5c154e30 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fdc5400e7c0 tx=0x7fdc5400ec80 comp rx=0 tx=0).stop 2023-10-16T08:39:01.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.803+0000 7fdc63512640 1 -- 172.21.15.40:0/3052737576 shutdown_connections 2023-10-16T08:39:01.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.803+0000 7fdc63512640 1 --2- 172.21.15.40:0/3052737576 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdc400664a0 0x7fdc40068960 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:01.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.803+0000 7fdc63512640 1 --2- 172.21.15.40:0/3052737576 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdc5c15b040 0x7fdc5c154e30 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:01.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.803+0000 7fdc63512640 1 --2- 172.21.15.40:0/3052737576 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdc5c0adec0 0x7fdc5c0b1580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:01.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.803+0000 7fdc63512640 1 --2- 172.21.15.40:0/3052737576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc5c06c7c0 0x7fdc5c155680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:01.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.803+0000 7fdc63512640 1 -- 172.21.15.40:0/3052737576 >> 172.21.15.40:0/3052737576 conn(0x7fdc5c0ce960 msgr2=0x7fdc5c15a360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:01.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.803+0000 7fdc63512640 1 -- 172.21.15.40:0/3052737576 shutdown_connections 2023-10-16T08:39:01.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:01.803+0000 7fdc63512640 1 -- 172.21.15.40:0/3052737576 wait complete. 2023-10-16T08:39:01.825 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T08:39:01.825 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T08:39:01.825 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi040.front.sepia.ceph.com 2023-10-16T08:39:01.826 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-16T08:39:01.826 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-16T08:39:01.826 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T08:39:01.826 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T08:39:01.827 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T08:39:01.827 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-16T08:39:01.827 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:01.827 DEBUG:teuthology.orchestra.run.smithi040:> ip addr 2023-10-16T08:39:01.833 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:39:01.833 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:39:01.833 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:39:01.833 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:01.834 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-10-16T08:39:01.834 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:01.834 INFO:teuthology.orchestra.run.smithi040.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:39:01.834 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:16 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:39:01.834 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f0 2023-10-16T08:39:01.835 INFO:teuthology.orchestra.run.smithi040.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:39:01.835 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:6b:fd:17 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:39:01.835 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp7s0f1 2023-10-16T08:39:01.835 INFO:teuthology.orchestra.run.smithi040.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T08:39:01.835 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cc brd ff:ff:ff:ff:ff:ff 2023-10-16T08:39:01.836 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f0 2023-10-16T08:39:01.836 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.21.15.40/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-10-16T08:39:01.836 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft 42484sec preferred_lft 42484sec 2023-10-16T08:39:01.836 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::ec4:7aff:febd:12cc/64 scope link 2023-10-16T08:39:01.836 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:01.836 INFO:teuthology.orchestra.run.smithi040.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:39:01.837 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0c:c4:7a:bd:12:cd brd ff:ff:ff:ff:ff:ff 2023-10-16T08:39:01.837 INFO:teuthology.orchestra.run.smithi040.stdout: altname enp1s0f1 2023-10-16T08:39:01.837 INFO:teuthology.orchestra.run.smithi040.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T08:39:01.837 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 02:42:65:e1:22:0d brd ff:ff:ff:ff:ff:ff 2023-10-16T08:39:01.837 INFO:teuthology.orchestra.run.smithi040.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T08:39:01.838 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:01.838 INFO:teuthology.orchestra.run.smithi040.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T08:39:01.838 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-16T08:39:01.838 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T08:39:01.838 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:01.839 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-16T08:39:01.839 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:01.839 INFO:teuthology.orchestra.run.smithi040.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T08:39:01.839 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:39:01.839 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-16T08:39:01.839 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:01.840 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:01.840 DEBUG:teuthology.orchestra.run.smithi040:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T08:39:01.885 INFO:teuthology.orchestra.run.smithi040.stdout:1 2023-10-16T08:39:01.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:01.887 DEBUG:teuthology.orchestra.run.smithi040:> route 2023-10-16T08:39:01.937 INFO:teuthology.orchestra.run.smithi040.stdout:Kernel IP routing table 2023-10-16T08:39:01.937 INFO:teuthology.orchestra.run.smithi040.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T08:39:01.937 INFO:teuthology.orchestra.run.smithi040.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-10-16T08:39:01.937 INFO:teuthology.orchestra.run.smithi040.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T08:39:01.938 INFO:teuthology.orchestra.run.smithi040.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-10-16T08:39:01.938 INFO:teuthology.orchestra.run.smithi040.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T08:39:01.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:01.938 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-16T08:39:01.939 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-16T08:39:01.939 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-10-16T08:39:01.939 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-10-16T08:39:01.940 DEBUG:teuthology.orchestra.run.smithi040:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-10-16T08:39:01.940 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-16T08:39:02.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:02.017 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-10-16T08:39:02.064 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T08:39:02.065 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:02.066 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list-id 2023-10-16T08:39:02.112 INFO:teuthology.orchestra.run.smithi040.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-16T08:39:02.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:02.113 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-16T08:39:02.113 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-16T08:39:02.113 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:39:02.114 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-16T08:39:02.114 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-16T08:39:02.187 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:02.188 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-16T08:39:02.246 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:39:02.247 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:39:02.247 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:39:02.247 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:02.247 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-10-16T08:39:02.247 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:02.248 INFO:teuthology.orchestra.run.smithi040.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T08:39:02.248 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 96:06:40:8e:ad:0d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-16T08:39:02.248 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-16T08:39:02.248 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:02.248 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::9406:40ff:fe8e:ad0d/64 scope link 2023-10-16T08:39:02.249 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:02.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:02.250 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-16T08:39:02.302 INFO:teuthology.orchestra.run.smithi040.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:39:02.303 INFO:teuthology.orchestra.run.smithi040.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:39:02.303 INFO:teuthology.orchestra.run.smithi040.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:39:02.303 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:02.303 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 ::1/128 scope host 2023-10-16T08:39:02.304 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:02.304 INFO:teuthology.orchestra.run.smithi040.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T08:39:02.304 INFO:teuthology.orchestra.run.smithi040.stdout: link/ether 96:06:40:8e:ad:0d brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-16T08:39:02.304 INFO:teuthology.orchestra.run.smithi040.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-16T08:39:02.304 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:02.305 INFO:teuthology.orchestra.run.smithi040.stdout: inet6 fe80::9406:40ff:fe8e:ad0d/64 scope link 2023-10-16T08:39:02.305 INFO:teuthology.orchestra.run.smithi040.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:02.306 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-16T08:39:02.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:02.307 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-16T08:39:02.307 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-16T08:39:02.307 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-16T08:39:02.308 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-16T08:39:02.308 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-16T08:39:02.308 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-16T08:39:02.308 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-16T08:39:02.309 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-16T08:39:02.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:02.401 DEBUG:teuthology.orchestra.run.smithi040:> (cd / && exec bash -c ' 2023-10-16T08:39:02.402 DEBUG:teuthology.orchestra.run.smithi040:> set -e 2023-10-16T08:39:02.402 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set brx.1 up 2023-10-16T08:39:02.402 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link set dev brx.1 master ceph-brx 2023-10-16T08:39:02.402 DEBUG:teuthology.orchestra.run.smithi040:> ') 2023-10-16T08:39:02.423 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-10-16T08:39:02.423 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:39:02.423 DEBUG:teuthology.orchestra.run.smithi040:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-16T08:39:02.470 INFO:teuthology.orchestra.run.smithi040.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-16T08:39:02.471 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:39:02.471 DEBUG:teuthology.orchestra.run.smithi040:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-16T08:39:02.518 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2023-10-16T08:39:02.574 DEBUG:teuthology.orchestra.run.smithi040:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:39:02.626 INFO:teuthology.orchestra.run.smithi040.stdout:/sys 2023-10-16T08:39:02.626 INFO:teuthology.orchestra.run.smithi040.stdout:/proc 2023-10-16T08:39:02.626 INFO:teuthology.orchestra.run.smithi040.stdout:/dev 2023-10-16T08:39:02.627 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/pts 2023-10-16T08:39:02.627 INFO:teuthology.orchestra.run.smithi040.stdout:/run 2023-10-16T08:39:02.627 INFO:teuthology.orchestra.run.smithi040.stdout:/ 2023-10-16T08:39:02.628 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/security 2023-10-16T08:39:02.628 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/shm 2023-10-16T08:39:02.628 INFO:teuthology.orchestra.run.smithi040.stdout:/run/lock 2023-10-16T08:39:02.628 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/cgroup 2023-10-16T08:39:02.629 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/pstore 2023-10-16T08:39:02.629 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/bpf 2023-10-16T08:39:02.629 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:39:02.629 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/hugepages 2023-10-16T08:39:02.629 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/mqueue 2023-10-16T08:39:02.630 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/debug 2023-10-16T08:39:02.630 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/tracing 2023-10-16T08:39:02.630 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/fs/nfsd 2023-10-16T08:39:02.630 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/fuse/connections 2023-10-16T08:39:02.631 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/config 2023-10-16T08:39:02.631 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:39:02.631 INFO:teuthology.orchestra.run.smithi040.stdout:/run/rpc_pipefs 2023-10-16T08:39:02.631 INFO:teuthology.orchestra.run.smithi040.stdout:/run/user/1000 2023-10-16T08:39:02.632 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph 2023-10-16T08:39:02.632 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:39:02.632 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:39:02.632 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:39:02.633 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:39:02.633 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns 2023-10-16T08:39:02.633 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:39:02.633 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:39:02.633 INFO:teuthology.orchestra.run.smithi040.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-16T08:39:02.634 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:39:02.634 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:39:02.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:02.635 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2023-10-16T08:39:02.683 INFO:teuthology.orchestra.run.smithi040.stdout:45 2023-10-16T08:39:02.684 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2023-10-16T08:39:02.684 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-10-16T08:39:02.728 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2023-10-16T08:39:02.745 DEBUG:teuthology.orchestra.run.smithi040:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:39:02.796 INFO:teuthology.orchestra.run.smithi040.stdout:/sys 2023-10-16T08:39:02.796 INFO:teuthology.orchestra.run.smithi040.stdout:/proc 2023-10-16T08:39:02.797 INFO:teuthology.orchestra.run.smithi040.stdout:/dev 2023-10-16T08:39:02.797 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/pts 2023-10-16T08:39:02.797 INFO:teuthology.orchestra.run.smithi040.stdout:/run 2023-10-16T08:39:02.797 INFO:teuthology.orchestra.run.smithi040.stdout:/ 2023-10-16T08:39:02.797 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/security 2023-10-16T08:39:02.798 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/shm 2023-10-16T08:39:02.798 INFO:teuthology.orchestra.run.smithi040.stdout:/run/lock 2023-10-16T08:39:02.798 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/cgroup 2023-10-16T08:39:02.798 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/pstore 2023-10-16T08:39:02.798 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/bpf 2023-10-16T08:39:02.799 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:39:02.799 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/hugepages 2023-10-16T08:39:02.799 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/mqueue 2023-10-16T08:39:02.799 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/debug 2023-10-16T08:39:02.800 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/tracing 2023-10-16T08:39:02.800 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/fs/nfsd 2023-10-16T08:39:02.801 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/fuse/connections 2023-10-16T08:39:02.801 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/config 2023-10-16T08:39:02.801 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:39:02.802 INFO:teuthology.orchestra.run.smithi040.stdout:/run/rpc_pipefs 2023-10-16T08:39:02.802 INFO:teuthology.orchestra.run.smithi040.stdout:/run/user/1000 2023-10-16T08:39:02.802 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph 2023-10-16T08:39:02.802 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:39:02.803 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:39:02.803 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:39:02.803 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:39:02.803 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns 2023-10-16T08:39:02.803 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:39:02.804 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:39:02.804 INFO:teuthology.orchestra.run.smithi040.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-16T08:39:02.804 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:39:02.804 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:39:02.805 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:02.806 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2023-10-16T08:39:02.814 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi040.stderr:2023-10-16T08:39:02.807+0000 7f35fe67c580 -1 init, newargv = 0x55e0cf69bc30 newargc=15 2023-10-16T08:39:02.814 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi040.stderr:ceph-fuse[45215]: starting ceph client 2023-10-16T08:39:02.818 INFO:teuthology.orchestra.run.smithi040.stdout:45 2023-10-16T08:39:02.821 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi040.stderr:ceph-fuse[45215]: starting fuse 2023-10-16T08:39:03.920 DEBUG:teuthology.orchestra.run.smithi040:> sudo modprobe fuse 2023-10-16T08:39:03.937 DEBUG:teuthology.orchestra.run.smithi040:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:39:03.985 INFO:teuthology.orchestra.run.smithi040.stdout:/sys 2023-10-16T08:39:03.985 INFO:teuthology.orchestra.run.smithi040.stdout:/proc 2023-10-16T08:39:03.985 INFO:teuthology.orchestra.run.smithi040.stdout:/dev 2023-10-16T08:39:03.985 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/pts 2023-10-16T08:39:03.986 INFO:teuthology.orchestra.run.smithi040.stdout:/run 2023-10-16T08:39:03.986 INFO:teuthology.orchestra.run.smithi040.stdout:/ 2023-10-16T08:39:03.986 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/security 2023-10-16T08:39:03.986 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/shm 2023-10-16T08:39:03.986 INFO:teuthology.orchestra.run.smithi040.stdout:/run/lock 2023-10-16T08:39:03.987 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/cgroup 2023-10-16T08:39:03.987 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/pstore 2023-10-16T08:39:03.987 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/bpf 2023-10-16T08:39:03.987 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:39:03.987 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/hugepages 2023-10-16T08:39:03.988 INFO:teuthology.orchestra.run.smithi040.stdout:/dev/mqueue 2023-10-16T08:39:03.988 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/debug 2023-10-16T08:39:03.988 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/tracing 2023-10-16T08:39:03.988 INFO:teuthology.orchestra.run.smithi040.stdout:/proc/fs/nfsd 2023-10-16T08:39:03.988 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/fs/fuse/connections 2023-10-16T08:39:03.989 INFO:teuthology.orchestra.run.smithi040.stdout:/sys/kernel/config 2023-10-16T08:39:03.989 INFO:teuthology.orchestra.run.smithi040.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:39:03.989 INFO:teuthology.orchestra.run.smithi040.stdout:/run/rpc_pipefs 2023-10-16T08:39:03.989 INFO:teuthology.orchestra.run.smithi040.stdout:/run/user/1000 2023-10-16T08:39:03.989 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph 2023-10-16T08:39:03.990 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-0 2023-10-16T08:39:03.990 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-1 2023-10-16T08:39:03.990 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-2 2023-10-16T08:39:03.990 INFO:teuthology.orchestra.run.smithi040.stdout:/var/lib/ceph/osd/ceph-3 2023-10-16T08:39:03.990 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns 2023-10-16T08:39:03.991 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:39:03.991 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:39:03.991 INFO:teuthology.orchestra.run.smithi040.stdout:/home/ubuntu/cephtest/mnt.0 2023-10-16T08:39:03.991 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:39:03.991 INFO:teuthology.orchestra.run.smithi040.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:39:03.992 INFO:teuthology.orchestra.run.smithi040.stdout:/home/ubuntu/cephtest/mnt.2 2023-10-16T08:39:03.992 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:03.993 DEBUG:teuthology.orchestra.run.smithi040:> ls /sys/fs/fuse/connections 2023-10-16T08:39:04.039 INFO:teuthology.orchestra.run.smithi040.stdout:45 2023-10-16T08:39:04.039 INFO:teuthology.orchestra.run.smithi040.stdout:46 2023-10-16T08:39:04.040 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2023-10-16T08:39:04.040 DEBUG:teuthology.orchestra.run.smithi040:> sudo stdin-killer -- python3 -c ' 2023-10-16T08:39:04.041 DEBUG:teuthology.orchestra.run.smithi040:> import glob 2023-10-16T08:39:04.041 DEBUG:teuthology.orchestra.run.smithi040:> import re 2023-10-16T08:39:04.041 DEBUG:teuthology.orchestra.run.smithi040:> import os 2023-10-16T08:39:04.041 DEBUG:teuthology.orchestra.run.smithi040:> import subprocess 2023-10-16T08:39:04.042 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-16T08:39:04.042 DEBUG:teuthology.orchestra.run.smithi040:> def _find_admin_socket(client_name): 2023-10-16T08:39:04.042 DEBUG:teuthology.orchestra.run.smithi040:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-10-16T08:39:04.043 DEBUG:teuthology.orchestra.run.smithi040:> files = glob.glob(asok_path) 2023-10-16T08:39:04.043 DEBUG:teuthology.orchestra.run.smithi040:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-10-16T08:39:04.043 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-16T08:39:04.044 DEBUG:teuthology.orchestra.run.smithi040:> # Given a non-glob path, it better be there 2023-10-16T08:39:04.044 DEBUG:teuthology.orchestra.run.smithi040:> if "*" not in asok_path: 2023-10-16T08:39:04.044 DEBUG:teuthology.orchestra.run.smithi040:> assert(len(files) == 1) 2023-10-16T08:39:04.045 DEBUG:teuthology.orchestra.run.smithi040:> return files[0] 2023-10-16T08:39:04.045 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-16T08:39:04.045 DEBUG:teuthology.orchestra.run.smithi040:> for f in files: 2023-10-16T08:39:04.045 DEBUG:teuthology.orchestra.run.smithi040:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T08:39:04.046 DEBUG:teuthology.orchestra.run.smithi040:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T08:39:04.046 DEBUG:teuthology.orchestra.run.smithi040:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T08:39:04.046 DEBUG:teuthology.orchestra.run.smithi040:> contents = proc_f.read() 2023-10-16T08:39:04.047 DEBUG:teuthology.orchestra.run.smithi040:> if mountpoint in contents: 2023-10-16T08:39:04.047 DEBUG:teuthology.orchestra.run.smithi040:> return f 2023-10-16T08:39:04.047 DEBUG:teuthology.orchestra.run.smithi040:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T08:39:04.047 DEBUG:teuthology.orchestra.run.smithi040:> 2023-10-16T08:39:04.048 DEBUG:teuthology.orchestra.run.smithi040:> print(_find_admin_socket("client.2")) 2023-10-16T08:39:04.048 DEBUG:teuthology.orchestra.run.smithi040:> ' 2023-10-16T08:39:04.167 INFO:teuthology.orchestra.run.smithi040.stdout:/var/run/ceph/ceph-client.2.45215.asok 2023-10-16T08:39:04.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T08:39:04.179 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.45215.asok 2023-10-16T08:39:04.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:04.179 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.45215.asok status 2023-10-16T08:39:04.339 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-16T08:39:04.339 INFO:teuthology.orchestra.run.smithi040.stdout: "metadata": { 2023-10-16T08:39:04.339 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_sha1": "c7471068426370a3754c2250e7729280233d552b", 2023-10-16T08:39:04.340 INFO:teuthology.orchestra.run.smithi040.stdout: "ceph_version": "ceph version 18.0.0-6675-gc7471068 (c7471068426370a3754c2250e7729280233d552b) reef (dev)", 2023-10-16T08:39:04.340 INFO:teuthology.orchestra.run.smithi040.stdout: "entity_id": "2", 2023-10-16T08:39:04.340 INFO:teuthology.orchestra.run.smithi040.stdout: "hostname": "smithi040", 2023-10-16T08:39:04.340 INFO:teuthology.orchestra.run.smithi040.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-10-16T08:39:04.340 INFO:teuthology.orchestra.run.smithi040.stdout: "pid": "45215", 2023-10-16T08:39:04.341 INFO:teuthology.orchestra.run.smithi040.stdout: "root": "/" 2023-10-16T08:39:04.341 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-16T08:39:04.341 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_count": 0, 2023-10-16T08:39:04.341 INFO:teuthology.orchestra.run.smithi040.stdout: "dentry_pinned_count": 0, 2023-10-16T08:39:04.341 INFO:teuthology.orchestra.run.smithi040.stdout: "id": 4686, 2023-10-16T08:39:04.341 INFO:teuthology.orchestra.run.smithi040.stdout: "inst": { 2023-10-16T08:39:04.342 INFO:teuthology.orchestra.run.smithi040.stdout: "name": { 2023-10-16T08:39:04.342 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "client", 2023-10-16T08:39:04.342 INFO:teuthology.orchestra.run.smithi040.stdout: "num": 4686 2023-10-16T08:39:04.342 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-16T08:39:04.342 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2023-10-16T08:39:04.343 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-16T08:39:04.343 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "192.168.0.2:0", 2023-10-16T08:39:04.343 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3158636446 2023-10-16T08:39:04.343 INFO:teuthology.orchestra.run.smithi040.stdout: } 2023-10-16T08:39:04.343 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-16T08:39:04.344 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": { 2023-10-16T08:39:04.344 INFO:teuthology.orchestra.run.smithi040.stdout: "type": "v1", 2023-10-16T08:39:04.344 INFO:teuthology.orchestra.run.smithi040.stdout: "addr": "192.168.0.2:0", 2023-10-16T08:39:04.344 INFO:teuthology.orchestra.run.smithi040.stdout: "nonce": 3158636446 2023-10-16T08:39:04.345 INFO:teuthology.orchestra.run.smithi040.stdout: }, 2023-10-16T08:39:04.345 INFO:teuthology.orchestra.run.smithi040.stdout: "inst_str": "client.4686 192.168.0.2:0/3158636446", 2023-10-16T08:39:04.346 INFO:teuthology.orchestra.run.smithi040.stdout: "addr_str": "192.168.0.2:0/3158636446", 2023-10-16T08:39:04.346 INFO:teuthology.orchestra.run.smithi040.stdout: "inode_count": 1, 2023-10-16T08:39:04.346 INFO:teuthology.orchestra.run.smithi040.stdout: "mds_epoch": 6, 2023-10-16T08:39:04.346 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch": 20, 2023-10-16T08:39:04.346 INFO:teuthology.orchestra.run.smithi040.stdout: "osd_epoch_barrier": 0, 2023-10-16T08:39:04.346 INFO:teuthology.orchestra.run.smithi040.stdout: "blocklisted": false, 2023-10-16T08:39:04.347 INFO:teuthology.orchestra.run.smithi040.stdout: "fs_name": "cephfs" 2023-10-16T08:39:04.347 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-16T08:39:04.354 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T08:39:04.478 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.471+0000 7f6dadb89640 1 Processor -- start 2023-10-16T08:39:04.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.471+0000 7f6dadb89640 1 -- start start 2023-10-16T08:39:04.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.475+0000 7f6dadb89640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6da81648d0 0x7f6da80cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:04.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.475+0000 7f6dadb89640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6da806c550 0x7f6da80ce1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:04.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.475+0000 7f6dadb89640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da8165c90 con 0x7f6da80c15f0 2023-10-16T08:39:04.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.475+0000 7f6dadb89640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da8165080 con 0x7f6da806c550 2023-10-16T08:39:04.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.475+0000 7f6dadb89640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da8165360 con 0x7f6da81648d0 2023-10-16T08:39:04.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.475+0000 7f6da7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6da806c550 0x7f6da80ce1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:04.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.475+0000 7f6da7fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6da806c550 0x7f6da80ce1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34230/0 (socket says 172.21.15.40:34230) 2023-10-16T08:39:04.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.475+0000 7f6da7fff640 1 -- 172.21.15.40:0/4279251173 learned_addr learned my addr 172.21.15.40:0/4279251173 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:04.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.475+0000 7f6da6ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6da80c15f0 0x7f6da80cc530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:50438/0 (socket says 172.21.15.40:50438) 2023-10-16T08:39:04.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.475+0000 7f6da77fe640 1 --2- 172.21.15.40:0/4279251173 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6da81648d0 0x7f6da80cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:04.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.475+0000 7f6da7fff640 1 -- 172.21.15.40:0/4279251173 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6da81648d0 msgr2=0x7f6da80cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:04.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.475+0000 7f6da7fff640 1 --2- 172.21.15.40:0/4279251173 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6da81648d0 0x7f6da80cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:04.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.475+0000 7f6da7fff640 1 -- 172.21.15.40:0/4279251173 >> v1:172.21.15.40:6789/0 conn(0x7f6da80c15f0 legacy=0x7f6da80cc530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:04.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.475+0000 7f6da7fff640 1 -- 172.21.15.40:0/4279251173 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da8165640 con 0x7f6da806c550 2023-10-16T08:39:04.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.475+0000 7f6da7fff640 1 --2- 172.21.15.40:0/4279251173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da806c550 0x7f6da80ce1e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009e50 tx=0x7f6da001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43faa53568ff2ca5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:04.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6da67fc640 1 -- 172.21.15.40:0/4279251173 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da00a5720 con 0x7f6da806c550 2023-10-16T08:39:04.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6da67fc640 1 -- 172.21.15.40:0/4279251173 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da00a5880 con 0x7f6da806c550 2023-10-16T08:39:04.484 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6da67fc640 1 -- 172.21.15.40:0/4279251173 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da00a5c10 con 0x7f6da806c550 2023-10-16T08:39:04.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6dadb89640 1 -- 172.21.15.40:0/4279251173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da806c550 msgr2=0x7f6da80ce1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:04.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6dadb89640 1 --2- 172.21.15.40:0/4279251173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da806c550 0x7f6da80ce1e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f6da0009e50 tx=0x7f6da001c950 comp rx=0 tx=0).stop 2023-10-16T08:39:04.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6dadb89640 1 -- 172.21.15.40:0/4279251173 shutdown_connections 2023-10-16T08:39:04.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6dadb89640 1 --2- 172.21.15.40:0/4279251173 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da806c550 0x7f6da80ce1e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:04.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6dadb89640 1 --2- 172.21.15.40:0/4279251173 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6da81648d0 0x7f6da80cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:04.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6dadb89640 1 -- 172.21.15.40:0/4279251173 >> 172.21.15.40:0/4279251173 conn(0x7f6da80ce960 msgr2=0x7f6da80ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:04.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6dadb89640 1 -- 172.21.15.40:0/4279251173 shutdown_connections 2023-10-16T08:39:04.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6dadb89640 1 -- 172.21.15.40:0/4279251173 wait complete. 2023-10-16T08:39:04.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6dadb89640 1 Processor -- start 2023-10-16T08:39:04.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6dadb89640 1 -- start start 2023-10-16T08:39:04.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6dadb89640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da806c550 0x7f6da81539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:04.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6dadb89640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6da80c15f0 0x7f6da8160af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:04.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6dadb89640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da81648d0 0x7f6da8161030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:04.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6dadb89640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da8159860 con 0x7f6da81648d0 2023-10-16T08:39:04.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6dadb89640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da815b110 con 0x7f6da806c550 2023-10-16T08:39:04.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6dadb89640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da815b3f0 con 0x7f6da80c15f0 2023-10-16T08:39:04.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6da7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da81648d0 0x7f6da8161030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:04.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6da7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da81648d0 0x7f6da8161030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34242/0 (socket says 172.21.15.40:34242) 2023-10-16T08:39:04.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6da7fff640 1 -- 172.21.15.40:0/512075472 learned_addr learned my addr 172.21.15.40:0/512075472 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:04.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6da6ffd640 1 --2- 172.21.15.40:0/512075472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6da80c15f0 0x7f6da8160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:04.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6da77fe640 1 --2- 172.21.15.40:0/512075472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da806c550 0x7f6da81539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:04.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6da7fff640 1 -- 172.21.15.40:0/512075472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6da80c15f0 msgr2=0x7f6da8160af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:04.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6da7fff640 1 --2- 172.21.15.40:0/512075472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6da80c15f0 0x7f6da8160af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:04.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6da7fff640 1 -- 172.21.15.40:0/512075472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da806c550 msgr2=0x7f6da81539f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:04.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6da7fff640 1 --2- 172.21.15.40:0/512075472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da806c550 0x7f6da81539f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:04.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6da7fff640 1 -- 172.21.15.40:0/512075472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da0009b30 con 0x7f6da81648d0 2023-10-16T08:39:04.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6da77fe640 1 --2- 172.21.15.40:0/512075472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da806c550 0x7f6da81539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:04.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6da7fff640 1 --2- 172.21.15.40:0/512075472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da81648d0 0x7f6da8161030 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6da001a580 tx=0x7f6da00196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:04.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6d8ffff640 1 -- 172.21.15.40:0/512075472 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da00b8470 con 0x7f6da81648d0 2023-10-16T08:39:04.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6d8ffff640 1 -- 172.21.15.40:0/512075472 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6da00b85d0 con 0x7f6da81648d0 2023-10-16T08:39:04.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.479+0000 7f6dadb89640 1 -- 172.21.15.40:0/512075472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da815b6d0 con 0x7f6da81648d0 2023-10-16T08:39:04.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.483+0000 7f6d8ffff640 1 -- 172.21.15.40:0/512075472 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6da00b8980 con 0x7f6da81648d0 2023-10-16T08:39:04.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.483+0000 7f6dadb89640 1 -- 172.21.15.40:0/512075472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da815a3c0 con 0x7f6da81648d0 2023-10-16T08:39:04.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.483+0000 7f6dadb89640 1 -- 172.21.15.40:0/512075472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da804e9c0 con 0x7f6da81648d0 2023-10-16T08:39:04.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.483+0000 7f6d8ffff640 1 -- 172.21.15.40:0/512075472 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6da00b8ae0 con 0x7f6da81648d0 2023-10-16T08:39:04.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.483+0000 7f6d8ffff640 1 --2- 172.21.15.40:0/512075472 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f6d940664a0 0x7f6d94068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:04.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.483+0000 7f6d8ffff640 1 -- 172.21.15.40:0/512075472 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6da011efc0 con 0x7f6da81648d0 2023-10-16T08:39:04.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.483+0000 7f6da77fe640 1 --2- 172.21.15.40:0/512075472 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f6d940664a0 0x7f6d94068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:04.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.483+0000 7f6da77fe640 1 --2- 172.21.15.40:0/512075472 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f6d940664a0 0x7f6d94068960 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6da81fd040 tx=0x7f6d98008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:04.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.487+0000 7f6d8ffff640 1 -- 172.21.15.40:0/512075472 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f6da00ec560 con 0x7f6da81648d0 2023-10-16T08:39:04.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.679+0000 7f6dadb89640 1 -- 172.21.15.40:0/512075472 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f6da80adc80 con 0x7f6da81648d0 2023-10-16T08:39:04.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.683+0000 7f6d8ffff640 1 -- 172.21.15.40:0/512075472 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6da00efb80 con 0x7f6da81648d0 2023-10-16T08:39:04.687 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T08:39:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.683+0000 7f6dadb89640 1 -- 172.21.15.40:0/512075472 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f6d940664a0 msgr2=0x7f6d94068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:04.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.683+0000 7f6dadb89640 1 --2- 172.21.15.40:0/512075472 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f6d940664a0 0x7f6d94068960 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6da81fd040 tx=0x7f6d98008040 comp rx=0 tx=0).stop 2023-10-16T08:39:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.683+0000 7f6dadb89640 1 -- 172.21.15.40:0/512075472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da81648d0 msgr2=0x7f6da8161030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.683+0000 7f6dadb89640 1 --2- 172.21.15.40:0/512075472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da81648d0 0x7f6da8161030 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f6da001a580 tx=0x7f6da00196f0 comp rx=0 tx=0).stop 2023-10-16T08:39:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.683+0000 7f6dadb89640 1 -- 172.21.15.40:0/512075472 shutdown_connections 2023-10-16T08:39:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.683+0000 7f6dadb89640 1 --2- 172.21.15.40:0/512075472 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f6d940664a0 0x7f6d94068960 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:04.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.683+0000 7f6dadb89640 1 --2- 172.21.15.40:0/512075472 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6da81648d0 0x7f6da8161030 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.683+0000 7f6dadb89640 1 --2- 172.21.15.40:0/512075472 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6da80c15f0 0x7f6da8160af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.683+0000 7f6dadb89640 1 --2- 172.21.15.40:0/512075472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6da806c550 0x7f6da81539f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.683+0000 7f6dadb89640 1 -- 172.21.15.40:0/512075472 >> 172.21.15.40:0/512075472 conn(0x7f6da80ce960 msgr2=0x7f6da80ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.683+0000 7f6dadb89640 1 -- 172.21.15.40:0/512075472 shutdown_connections 2023-10-16T08:39:04.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:04.683+0000 7f6dadb89640 1 -- 172.21.15.40:0/512075472 wait complete. 2023-10-16T08:39:04.706 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T08:39:04.707 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T08:39:04.707 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2023-10-16T08:39:04.707 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-16T08:39:04.707 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-16T08:39:04.708 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T08:39:04.708 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T08:39:04.708 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T08:39:04.708 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-16T08:39:04.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:04.709 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2023-10-16T08:39:04.716 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:39:04.716 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:39:04.716 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:39:04.717 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:04.717 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2023-10-16T08:39:04.717 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:04.717 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:39:04.717 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:39:04.718 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2023-10-16T08:39:04.718 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:39:04.718 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:39:04.718 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2023-10-16T08:39:04.718 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:39:04.719 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:39:04.719 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T08:39:04.719 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:39:04.719 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-16T08:39:04.719 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42473sec preferred_lft 42473sec 2023-10-16T08:39:04.720 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2023-10-16T08:39:04.720 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:04.720 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T08:39:04.720 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:f8:30:32:0e brd ff:ff:ff:ff:ff:ff 2023-10-16T08:39:04.720 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T08:39:04.721 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:04.721 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-16T08:39:04.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:04.722 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-16T08:39:04.722 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-16T08:39:04.722 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add name ceph-brx type bridge 2023-10-16T08:39:04.723 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr flush dev ceph-brx 2023-10-16T08:39:04.723 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set ceph-brx up 2023-10-16T08:39:04.723 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-16T08:39:04.724 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-16T08:39:04.799 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:04.800 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T08:39:04.850 INFO:teuthology.orchestra.run.smithi086.stdout:1 2023-10-16T08:39:04.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:04.852 DEBUG:teuthology.orchestra.run.smithi086:> route 2023-10-16T08:39:04.921 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2023-10-16T08:39:04.921 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T08:39:04.921 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-16T08:39:04.922 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T08:39:04.922 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T08:39:04.922 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T08:39:04.922 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T08:39:04.922 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T08:39:04.923 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T08:39:04.923 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T08:39:04.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:04.924 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-16T08:39:04.924 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-16T08:39:04.924 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T08:39:04.924 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T08:39:04.925 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T08:39:04.925 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-16T08:39:04.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:04.993 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2023-10-16T08:39:05.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:05.040 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2023-10-16T08:39:05.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:05.087 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-16T08:39:05.088 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-16T08:39:05.088 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:39:05.088 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-16T08:39:05.088 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-16T08:39:05.156 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-16T08:39:05.156 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:05.157 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-16T08:39:05.157 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-16T08:39:05.157 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-16T08:39:05.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-16T08:39:05.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-16T08:39:05.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-16T08:39:05.159 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-16T08:39:05.159 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-16T08:39:05.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:05.257 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-16T08:39:05.258 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-16T08:39:05.258 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2023-10-16T08:39:05.258 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.0 master ceph-brx 2023-10-16T08:39:05.259 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-16T08:39:05.277 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-16T08:39:05.277 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:39:05.277 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-16T08:39:05.322 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-16T08:39:05.322 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:39:05.323 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-16T08:39:05.370 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2023-10-16T08:39:05.428 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:39:05.477 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-16T08:39:05.478 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-16T08:39:05.478 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-16T08:39:05.478 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-16T08:39:05.479 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-16T08:39:05.479 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-16T08:39:05.480 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-16T08:39:05.480 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-16T08:39:05.480 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-16T08:39:05.481 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-16T08:39:05.481 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-16T08:39:05.481 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-16T08:39:05.482 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:39:05.482 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-16T08:39:05.482 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-16T08:39:05.483 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-16T08:39:05.483 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-16T08:39:05.483 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-16T08:39:05.484 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-16T08:39:05.484 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-16T08:39:05.484 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:39:05.485 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-16T08:39:05.485 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-16T08:39:05.486 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-16T08:39:05.486 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-16T08:39:05.486 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-16T08:39:05.487 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-16T08:39:05.487 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-16T08:39:05.487 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-16T08:39:05.487 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:39:05.487 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:39:05.489 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:05.489 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-16T08:39:05.534 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-16T08:39:05.535 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-16T08:39:05.584 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2023-10-16T08:39:05.598 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:39:05.651 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-16T08:39:05.651 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-16T08:39:05.651 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-16T08:39:05.651 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-16T08:39:05.652 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-16T08:39:05.652 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-16T08:39:05.652 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-16T08:39:05.652 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-16T08:39:05.652 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-16T08:39:05.653 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-16T08:39:05.653 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-16T08:39:05.653 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-16T08:39:05.653 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:39:05.653 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-16T08:39:05.654 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-16T08:39:05.654 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-16T08:39:05.654 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-16T08:39:05.654 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-16T08:39:05.654 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-16T08:39:05.654 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-16T08:39:05.655 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:39:05.655 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-16T08:39:05.655 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-16T08:39:05.655 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-16T08:39:05.655 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-16T08:39:05.656 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-16T08:39:05.656 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-16T08:39:05.656 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-16T08:39:05.656 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-16T08:39:05.656 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:39:05.657 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:39:05.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:05.658 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-16T08:39:05.671 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:2023-10-16T08:39:05.668+0000 7f2b4496c580 -1 init, newargv = 0x55a9a97e6f90 newargc=15 2023-10-16T08:39:05.671 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[44060]: starting ceph client 2023-10-16T08:39:05.679 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[44060]: starting fuse 2023-10-16T08:39:06.777 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2023-10-16T08:39:06.793 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:39:06.841 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-16T08:39:06.841 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-16T08:39:06.841 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-16T08:39:06.841 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-16T08:39:06.842 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-16T08:39:06.842 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-16T08:39:06.842 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-16T08:39:06.842 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-16T08:39:06.843 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-16T08:39:06.843 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-16T08:39:06.843 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-16T08:39:06.843 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-16T08:39:06.843 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:39:06.843 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-16T08:39:06.844 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-16T08:39:06.844 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-16T08:39:06.844 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-16T08:39:06.844 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-16T08:39:06.844 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-16T08:39:06.845 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-16T08:39:06.845 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:39:06.845 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-16T08:39:06.845 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-16T08:39:06.845 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-16T08:39:06.846 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-16T08:39:06.846 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-16T08:39:06.846 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-16T08:39:06.846 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-16T08:39:06.846 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-16T08:39:06.847 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:39:06.847 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:39:06.847 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-16T08:39:06.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:06.848 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-16T08:39:06.899 INFO:teuthology.orchestra.run.smithi086.stdout:45 2023-10-16T08:39:06.899 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45] 2023-10-16T08:39:06.900 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2023-10-16T08:39:06.900 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2023-10-16T08:39:06.900 DEBUG:teuthology.orchestra.run.smithi086:> import re 2023-10-16T08:39:06.901 DEBUG:teuthology.orchestra.run.smithi086:> import os 2023-10-16T08:39:06.901 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2023-10-16T08:39:06.901 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-16T08:39:06.902 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2023-10-16T08:39:06.902 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-16T08:39:06.902 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2023-10-16T08:39:06.903 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-16T08:39:06.903 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-16T08:39:06.903 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2023-10-16T08:39:06.904 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2023-10-16T08:39:06.904 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2023-10-16T08:39:06.905 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2023-10-16T08:39:06.905 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-16T08:39:06.906 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2023-10-16T08:39:06.906 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T08:39:06.907 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T08:39:06.908 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T08:39:06.908 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2023-10-16T08:39:06.909 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2023-10-16T08:39:06.909 DEBUG:teuthology.orchestra.run.smithi086:> return f 2023-10-16T08:39:06.909 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T08:39:06.910 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-16T08:39:06.911 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.1")) 2023-10-16T08:39:06.911 DEBUG:teuthology.orchestra.run.smithi086:> ' 2023-10-16T08:39:07.023 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.1.44060.asok 2023-10-16T08:39:07.027 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:39:07 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T08:39:07.035 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.44060.asok 2023-10-16T08:39:07.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:07.036 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.44060.asok status 2023-10-16T08:39:07.186 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2023-10-16T08:39:07.187 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2023-10-16T08:39:07.187 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "c7471068426370a3754c2250e7729280233d552b", 2023-10-16T08:39:07.187 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 18.0.0-6675-gc7471068 (c7471068426370a3754c2250e7729280233d552b) reef (dev)", 2023-10-16T08:39:07.187 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "1", 2023-10-16T08:39:07.187 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2023-10-16T08:39:07.188 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-16T08:39:07.188 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "44060", 2023-10-16T08:39:07.188 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2023-10-16T08:39:07.188 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-16T08:39:07.189 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2023-10-16T08:39:07.189 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2023-10-16T08:39:07.189 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 4481, 2023-10-16T08:39:07.189 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2023-10-16T08:39:07.189 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2023-10-16T08:39:07.189 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2023-10-16T08:39:07.189 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 4481 2023-10-16T08:39:07.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-16T08:39:07.190 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-16T08:39:07.190 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-16T08:39:07.190 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-16T08:39:07.190 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2211120169 2023-10-16T08:39:07.191 INFO:teuthology.orchestra.run.smithi086.stdout: } 2023-10-16T08:39:07.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-16T08:39:07.191 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-16T08:39:07.191 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-16T08:39:07.191 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2023-10-16T08:39:07.191 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2211120169 2023-10-16T08:39:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-16T08:39:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.4481 192.168.0.1:0/2211120169", 2023-10-16T08:39:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/2211120169", 2023-10-16T08:39:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2023-10-16T08:39:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 6, 2023-10-16T08:39:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 20, 2023-10-16T08:39:07.192 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2023-10-16T08:39:07.193 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2023-10-16T08:39:07.193 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2023-10-16T08:39:07.193 INFO:teuthology.orchestra.run.smithi086.stdout:} 2023-10-16T08:39:07.200 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-16T08:39:07.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.331+0000 7f64d7bdc640 1 Processor -- start 2023-10-16T08:39:07.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.331+0000 7f64d7bdc640 1 -- start start 2023-10-16T08:39:07.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.331+0000 7f64d7bdc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f64d00cbb90 0x7f64d015d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:07.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.331+0000 7f64d7bdc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d01617c0 con 0x7f64d0066190 2023-10-16T08:39:07.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.331+0000 7f64d7bdc640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d0160a00 con 0x7f64d015ceb0 2023-10-16T08:39:07.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.331+0000 7f64d7bdc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d0160ce0 con 0x7f64d00cbb90 2023-10-16T08:39:07.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.331+0000 7f64d6179640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f64d00cbb90 0x7f64d015d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:07.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.331+0000 7f64d5177640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f64d0066190 0x7f64d015d290 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54660/0 (socket says 172.21.15.40:54660) 2023-10-16T08:39:07.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.331+0000 7f64d5177640 1 -- 172.21.15.40:0/2509409707 learned_addr learned my addr 172.21.15.40:0/2509409707 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:07.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.331+0000 7f64d6179640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f64d00cbb90 0x7f64d015d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48006/0 (socket says 172.21.15.40:48006) 2023-10-16T08:39:07.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.331+0000 7f64d4976640 1 -- 172.21.15.40:0/2509409707 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2371836023 0 0) 0x7f64d01617c0 con 0x7f64d0066190 2023-10-16T08:39:07.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.331+0000 7f64d4976640 1 -- 172.21.15.40:0/2509409707 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64bc003530 con 0x7f64d0066190 2023-10-16T08:39:07.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.331+0000 7f64d6179640 1 -- 172.21.15.40:0/2509409707 >> v1:172.21.15.86:6789/0 conn(0x7f64d015ceb0 legacy=0x7f64d0065a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:07.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.331+0000 7f64d6179640 1 -- 172.21.15.40:0/2509409707 >> v1:172.21.15.40:6789/0 conn(0x7f64d0066190 legacy=0x7f64d015d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:07.339 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.331+0000 7f64d6179640 1 -- 172.21.15.40:0/2509409707 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d0160fc0 con 0x7f64d00cbb90 2023-10-16T08:39:07.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d6179640 1 --2- 172.21.15.40:0/2509409707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d00cbb90 0x7f64d015d9a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f64c80097c0 tx=0x7f64c801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f303d18d3372e86 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:07.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d4976640 1 -- 172.21.15.40:0/2509409707 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c80a7070 con 0x7f64d00cbb90 2023-10-16T08:39:07.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d4976640 1 -- 172.21.15.40:0/2509409707 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c80ab430 con 0x7f64d00cbb90 2023-10-16T08:39:07.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d4976640 1 -- 172.21.15.40:0/2509409707 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c80ab7c0 con 0x7f64d00cbb90 2023-10-16T08:39:07.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d7bdc640 1 -- 172.21.15.40:0/2509409707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d00cbb90 msgr2=0x7f64d015d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:07.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d7bdc640 1 --2- 172.21.15.40:0/2509409707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d00cbb90 0x7f64d015d9a0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f64c80097c0 tx=0x7f64c801c410 comp rx=0 tx=0).stop 2023-10-16T08:39:07.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d7bdc640 1 -- 172.21.15.40:0/2509409707 shutdown_connections 2023-10-16T08:39:07.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d7bdc640 1 --2- 172.21.15.40:0/2509409707 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d00cbb90 0x7f64d015d9a0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:07.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d7bdc640 1 -- 172.21.15.40:0/2509409707 >> 172.21.15.40:0/2509409707 conn(0x7f64d00cc360 msgr2=0x7f64d00cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:07.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d7bdc640 1 -- 172.21.15.40:0/2509409707 shutdown_connections 2023-10-16T08:39:07.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d7bdc640 1 -- 172.21.15.40:0/2509409707 wait complete. 2023-10-16T08:39:07.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d7bdc640 1 Processor -- start 2023-10-16T08:39:07.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d7bdc640 1 -- start start 2023-10-16T08:39:07.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d7bdc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0066190 0x7f64d0158d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:07.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d7bdc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64d00cbb90 0x7f64d00b06c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:07.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d7bdc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f64d015ceb0 0x7f64d00b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:07.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d7bdc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00af620 con 0x7f64d0066190 2023-10-16T08:39:07.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d7bdc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00af900 con 0x7f64d00cbb90 2023-10-16T08:39:07.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d7bdc640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d00afbe0 con 0x7f64d015ceb0 2023-10-16T08:39:07.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d5978640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0066190 0x7f64d0158d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:07.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d5978640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0066190 0x7f64d0158d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48018/0 (socket says 172.21.15.40:48018) 2023-10-16T08:39:07.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.335+0000 7f64d5978640 1 -- 172.21.15.40:0/3432976846 learned_addr learned my addr 172.21.15.40:0/3432976846 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:07.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.339+0000 7f64d6179640 1 --2- 172.21.15.40:0/3432976846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f64d015ceb0 0x7f64d00b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:07.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.339+0000 7f64d5177640 1 --2- 172.21.15.40:0/3432976846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64d00cbb90 0x7f64d00b06c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:07.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.339+0000 7f64d5978640 1 -- 172.21.15.40:0/3432976846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f64d015ceb0 msgr2=0x7f64d00b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:07.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.339+0000 7f64d5978640 1 --2- 172.21.15.40:0/3432976846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f64d015ceb0 0x7f64d00b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:07.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.339+0000 7f64d5978640 1 -- 172.21.15.40:0/3432976846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64d00cbb90 msgr2=0x7f64d00b06c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:07.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.339+0000 7f64d5978640 1 --2- 172.21.15.40:0/3432976846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64d00cbb90 0x7f64d00b06c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:07.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.339+0000 7f64d5978640 1 -- 172.21.15.40:0/3432976846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c8009420 con 0x7f64d0066190 2023-10-16T08:39:07.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.339+0000 7f64d5177640 1 --2- 172.21.15.40:0/3432976846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64d00cbb90 0x7f64d00b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:07.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.339+0000 7f64d5978640 1 --2- 172.21.15.40:0/3432976846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0066190 0x7f64d0158d90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f64c400c410 tx=0x7f64c400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:07.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.339+0000 7f64c27fc640 1 -- 172.21.15.40:0/3432976846 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4011790 con 0x7f64d0066190 2023-10-16T08:39:07.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.339+0000 7f64c27fc640 1 -- 172.21.15.40:0/3432976846 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c4011910 con 0x7f64d0066190 2023-10-16T08:39:07.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.339+0000 7f64c27fc640 1 -- 172.21.15.40:0/3432976846 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c4011d00 con 0x7f64d0066190 2023-10-16T08:39:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.339+0000 7f64d7bdc640 1 -- 172.21.15.40:0/3432976846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d00afec0 con 0x7f64d0066190 2023-10-16T08:39:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.339+0000 7f64d7bdc640 1 -- 172.21.15.40:0/3432976846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d00af240 con 0x7f64d0066190 2023-10-16T08:39:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.339+0000 7f64d7bdc640 1 -- 172.21.15.40:0/3432976846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d004e9c0 con 0x7f64d0066190 2023-10-16T08:39:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.339+0000 7f64c27fc640 1 -- 172.21.15.40:0/3432976846 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f64c401c460 con 0x7f64d0066190 2023-10-16T08:39:07.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.339+0000 7f64c27fc640 1 --2- 172.21.15.40:0/3432976846 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f64b40664a0 0x7f64b4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:07.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.339+0000 7f64c27fc640 1 -- 172.21.15.40:0/3432976846 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f64c4083db0 con 0x7f64d0066190 2023-10-16T08:39:07.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.339+0000 7f64d5177640 1 --2- 172.21.15.40:0/3432976846 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f64b40664a0 0x7f64b4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:07.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.339+0000 7f64d5177640 1 --2- 172.21.15.40:0/3432976846 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f64b40664a0 0x7f64b4068960 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f64d0154640 tx=0x7f64cc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:07.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.343+0000 7f64c27fc640 1 -- 172.21.15.40:0/3432976846 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f64c4051230 con 0x7f64d0066190 2023-10-16T08:39:07.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.535+0000 7f64d7bdc640 1 -- 172.21.15.40:0/3432976846 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f64d00b0370 con 0x7f64d0066190 2023-10-16T08:39:07.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.539+0000 7f64c27fc640 1 -- 172.21.15.40:0/3432976846 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v6) v1 ==== 53+0+73 (secure 0 0 0) 0x7f64c4011e60 con 0x7f64d0066190 2023-10-16T08:39:07.543 INFO:teuthology.orchestra.run.smithi040.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-16T08:39:07.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.539+0000 7f64d7bdc640 1 -- 172.21.15.40:0/3432976846 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f64b40664a0 msgr2=0x7f64b4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:07.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.539+0000 7f64d7bdc640 1 --2- 172.21.15.40:0/3432976846 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f64b40664a0 0x7f64b4068960 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f64d0154640 tx=0x7f64cc009040 comp rx=0 tx=0).stop 2023-10-16T08:39:07.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.539+0000 7f64d7bdc640 1 -- 172.21.15.40:0/3432976846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0066190 msgr2=0x7f64d0158d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:07.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.539+0000 7f64d7bdc640 1 --2- 172.21.15.40:0/3432976846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0066190 0x7f64d0158d90 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f64c400c410 tx=0x7f64c400c8d0 comp rx=0 tx=0).stop 2023-10-16T08:39:07.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.539+0000 7f64d7bdc640 1 -- 172.21.15.40:0/3432976846 shutdown_connections 2023-10-16T08:39:07.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.539+0000 7f64d7bdc640 1 --2- 172.21.15.40:0/3432976846 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f64d015ceb0 0x7f64d00b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:07.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.539+0000 7f64d7bdc640 1 --2- 172.21.15.40:0/3432976846 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64d00cbb90 0x7f64d00b06c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:07.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.539+0000 7f64d7bdc640 1 --2- 172.21.15.40:0/3432976846 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f64b40664a0 0x7f64b4068960 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:07.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.539+0000 7f64d7bdc640 1 --2- 172.21.15.40:0/3432976846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f64d0066190 0x7f64d0158d90 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:07.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.539+0000 7f64d7bdc640 1 -- 172.21.15.40:0/3432976846 >> 172.21.15.40:0/3432976846 conn(0x7f64d00cc360 msgr2=0x7f64d00add20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:07.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.539+0000 7f64d7bdc640 1 -- 172.21.15.40:0/3432976846 shutdown_connections 2023-10-16T08:39:07.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:07.539+0000 7f64d7bdc640 1 -- 172.21.15.40:0/3432976846 wait complete. 2023-10-16T08:39:07.562 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-16T08:39:07.562 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-16T08:39:07.562 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2023-10-16T08:39:07.563 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-16T08:39:07.563 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-16T08:39:07.563 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-16T08:39:07.563 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-16T08:39:07.564 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-16T08:39:07.564 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-16T08:39:07.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:07.564 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2023-10-16T08:39:07.574 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:39:07.574 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:39:07.574 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:39:07.574 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:07.575 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2023-10-16T08:39:07.575 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:07.575 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:39:07.575 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:39:07.575 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2023-10-16T08:39:07.576 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:39:07.576 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:39:07.576 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2023-10-16T08:39:07.576 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-10-16T08:39:07.576 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:39:07.577 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-16T08:39:07.577 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2023-10-16T08:39:07.577 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 metric 100 brd 172.21.15.255 scope global dynamic enp3s0f1 2023-10-16T08:39:07.577 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42470sec preferred_lft 42470sec 2023-10-16T08:39:07.577 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ec4:7aff:fe8f:d809/64 scope link 2023-10-16T08:39:07.578 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:07.578 INFO:teuthology.orchestra.run.smithi086.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-10-16T08:39:07.578 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 02:42:f8:30:32:0e brd ff:ff:ff:ff:ff:ff 2023-10-16T08:39:07.578 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-10-16T08:39:07.578 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:07.579 INFO:teuthology.orchestra.run.smithi086.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T08:39:07.579 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-10-16T08:39:07.579 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-16T08:39:07.579 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:07.579 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-10-16T08:39:07.580 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:07.580 INFO:teuthology.orchestra.run.smithi086.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-16T08:39:07.580 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:39:07.580 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-10-16T08:39:07.580 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:07.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:07.581 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-16T08:39:07.633 INFO:teuthology.orchestra.run.smithi086.stdout:1 2023-10-16T08:39:07.635 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:07.635 DEBUG:teuthology.orchestra.run.smithi086:> route 2023-10-16T08:39:07.686 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2023-10-16T08:39:07.686 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-16T08:39:07.686 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 0 0 0 enp3s0f1 2023-10-16T08:39:07.687 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-16T08:39:07.687 INFO:teuthology.orchestra.run.smithi086.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-10-16T08:39:07.687 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-16T08:39:07.687 INFO:teuthology.orchestra.run.smithi086.stdout:ns1.front.sepia 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T08:39:07.688 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.2 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T08:39:07.688 INFO:teuthology.orchestra.run.smithi086.stdout:_gateway 0.0.0.0 255.255.255.255 UH 100 0 0 enp3s0f1 2023-10-16T08:39:07.688 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-16T08:39:07.688 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:07.689 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-16T08:39:07.689 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-16T08:39:07.689 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-16T08:39:07.690 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-16T08:39:07.690 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-16T08:39:07.690 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-16T08:39:07.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:07.763 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2023-10-16T08:39:07.811 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-16T08:39:07.811 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:07.812 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2023-10-16T08:39:07.858 INFO:teuthology.orchestra.run.smithi086.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-16T08:39:07.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:07.859 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-16T08:39:07.860 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-16T08:39:07.860 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T08:39:07.860 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-16T08:39:07.861 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-16T08:39:07.928 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:07.929 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-16T08:39:07.983 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:39:07.983 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:39:07.983 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:39:07.984 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:07.984 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2023-10-16T08:39:07.984 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:07.984 INFO:teuthology.orchestra.run.smithi086.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T08:39:07.984 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 1a:81:f1:8a:a3:ff brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-16T08:39:07.985 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-16T08:39:07.985 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:07.985 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::1881:f1ff:fe8a:a3ff/64 scope link 2023-10-16T08:39:07.985 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:07.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:07.987 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-16T08:39:08.042 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-16T08:39:08.043 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-16T08:39:08.043 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2023-10-16T08:39:08.043 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:08.043 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2023-10-16T08:39:08.044 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:08.044 INFO:teuthology.orchestra.run.smithi086.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-16T08:39:08.044 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 1a:81:f1:8a:a3:ff brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-16T08:39:08.044 INFO:teuthology.orchestra.run.smithi086.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-16T08:39:08.044 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:08.045 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::1881:f1ff:fe8a:a3ff/64 scope link 2023-10-16T08:39:08.045 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2023-10-16T08:39:08.046 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-16T08:39:08.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:08.046 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-16T08:39:08.047 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-16T08:39:08.047 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-16T08:39:08.047 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-16T08:39:08.047 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-16T08:39:08.048 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-16T08:39:08.048 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-16T08:39:08.048 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-16T08:39:08.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:08.145 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec bash -c ' 2023-10-16T08:39:08.146 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2023-10-16T08:39:08.146 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.1 up 2023-10-16T08:39:08.147 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.1 master ceph-brx 2023-10-16T08:39:08.147 DEBUG:teuthology.orchestra.run.smithi086:> ') 2023-10-16T08:39:08.168 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-10-16T08:39:08.169 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:39:08.169 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-16T08:39:08.214 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-16T08:39:08.215 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-16T08:39:08.215 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-16T08:39:08.261 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2023-10-16T08:39:08.320 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:39:08.369 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-16T08:39:08.369 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-16T08:39:08.369 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-16T08:39:08.370 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-16T08:39:08.370 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-16T08:39:08.370 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-16T08:39:08.370 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-16T08:39:08.370 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-16T08:39:08.371 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-16T08:39:08.371 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-16T08:39:08.371 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-16T08:39:08.371 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-16T08:39:08.371 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:39:08.372 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-16T08:39:08.372 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-16T08:39:08.372 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-16T08:39:08.372 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-16T08:39:08.372 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-16T08:39:08.373 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-16T08:39:08.373 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-16T08:39:08.373 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:39:08.373 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-16T08:39:08.373 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-16T08:39:08.374 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-16T08:39:08.374 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-16T08:39:08.374 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-16T08:39:08.374 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-16T08:39:08.374 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-16T08:39:08.375 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-16T08:39:08.375 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:39:08.375 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:39:08.375 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-16T08:39:08.375 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T08:39:08.375 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T08:39:08.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:08.377 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-16T08:39:08.423 INFO:teuthology.orchestra.run.smithi086.stdout:45 2023-10-16T08:39:08.423 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [45] 2023-10-16T08:39:08.424 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-10-16T08:39:08.468 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2023-10-16T08:39:08.483 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:39:08.530 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-16T08:39:08.531 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-16T08:39:08.531 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-16T08:39:08.531 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-16T08:39:08.531 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-16T08:39:08.531 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-16T08:39:08.531 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-16T08:39:08.532 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-16T08:39:08.532 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-16T08:39:08.532 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-16T08:39:08.532 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-16T08:39:08.532 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-16T08:39:08.533 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:39:08.533 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-16T08:39:08.533 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-16T08:39:08.533 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-16T08:39:08.533 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-16T08:39:08.533 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-16T08:39:08.534 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-16T08:39:08.534 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-16T08:39:08.534 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:39:08.534 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-16T08:39:08.534 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-16T08:39:08.535 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-16T08:39:08.535 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-16T08:39:08.535 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-16T08:39:08.535 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-16T08:39:08.535 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-16T08:39:08.536 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-16T08:39:08.536 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:39:08.536 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:39:08.536 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-16T08:39:08.536 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T08:39:08.536 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T08:39:08.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:08.538 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-16T08:39:08.543 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi086.stderr:2023-10-16T08:39:08.540+0000 7f4c3f9ea580 -1 init, newargv = 0x55bf20135010 newargc=15 2023-10-16T08:39:08.543 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi086.stderr:ceph-fuse[44175]: starting ceph client 2023-10-16T08:39:08.547 INFO:teuthology.orchestra.run.smithi086.stdout:45 2023-10-16T08:39:08.550 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi086.stderr:ceph-fuse[44175]: starting fuse 2023-10-16T08:39:09.649 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2023-10-16T08:39:09.665 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2023-10-16T08:39:09.719 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2023-10-16T08:39:09.720 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2023-10-16T08:39:09.720 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2023-10-16T08:39:09.720 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2023-10-16T08:39:09.721 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2023-10-16T08:39:09.721 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2023-10-16T08:39:09.721 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2023-10-16T08:39:09.721 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2023-10-16T08:39:09.721 INFO:teuthology.orchestra.run.smithi086.stdout:/run/lock 2023-10-16T08:39:09.722 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2023-10-16T08:39:09.722 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2023-10-16T08:39:09.722 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2023-10-16T08:39:09.722 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2023-10-16T08:39:09.722 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2023-10-16T08:39:09.723 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2023-10-16T08:39:09.723 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2023-10-16T08:39:09.723 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2023-10-16T08:39:09.723 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/fs/nfsd 2023-10-16T08:39:09.723 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2023-10-16T08:39:09.724 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2023-10-16T08:39:09.724 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2023-10-16T08:39:09.724 INFO:teuthology.orchestra.run.smithi086.stdout:/run/rpc_pipefs 2023-10-16T08:39:09.724 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2023-10-16T08:39:09.724 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2023-10-16T08:39:09.725 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-4 2023-10-16T08:39:09.725 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-5 2023-10-16T08:39:09.725 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-6 2023-10-16T08:39:09.725 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph/osd/ceph-7 2023-10-16T08:39:09.725 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2023-10-16T08:39:09.725 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:39:09.726 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:39:09.726 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.1 2023-10-16T08:39:09.726 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T08:39:09.726 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T08:39:09.726 INFO:teuthology.orchestra.run.smithi086.stdout:/home/ubuntu/cephtest/mnt.3 2023-10-16T08:39:09.727 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:09.728 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2023-10-16T08:39:09.774 INFO:teuthology.orchestra.run.smithi086.stdout:45 2023-10-16T08:39:09.775 INFO:teuthology.orchestra.run.smithi086.stdout:46 2023-10-16T08:39:09.775 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [45, 46] 2023-10-16T08:39:09.775 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2023-10-16T08:39:09.776 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2023-10-16T08:39:09.776 DEBUG:teuthology.orchestra.run.smithi086:> import re 2023-10-16T08:39:09.776 DEBUG:teuthology.orchestra.run.smithi086:> import os 2023-10-16T08:39:09.776 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2023-10-16T08:39:09.777 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-16T08:39:09.777 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2023-10-16T08:39:09.777 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-10-16T08:39:09.777 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2023-10-16T08:39:09.778 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-10-16T08:39:09.778 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-16T08:39:09.778 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2023-10-16T08:39:09.779 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2023-10-16T08:39:09.779 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2023-10-16T08:39:09.779 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2023-10-16T08:39:09.780 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-16T08:39:09.780 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2023-10-16T08:39:09.780 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-16T08:39:09.781 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-16T08:39:09.781 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-16T08:39:09.781 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2023-10-16T08:39:09.782 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2023-10-16T08:39:09.782 DEBUG:teuthology.orchestra.run.smithi086:> return f 2023-10-16T08:39:09.782 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-16T08:39:09.783 DEBUG:teuthology.orchestra.run.smithi086:> 2023-10-16T08:39:09.783 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.3")) 2023-10-16T08:39:09.783 DEBUG:teuthology.orchestra.run.smithi086:> ' 2023-10-16T08:39:09.897 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.3.44175.asok 2023-10-16T08:39:09.901 INFO:teuthology.orchestra.run.smithi086.stderr:2023-10-16T08:39:09 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2023-10-16T08:39:09.908 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.44175.asok 2023-10-16T08:39:09.909 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:09.909 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.44175.asok status 2023-10-16T08:39:10.063 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2023-10-16T08:39:10.064 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2023-10-16T08:39:10.064 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "c7471068426370a3754c2250e7729280233d552b", 2023-10-16T08:39:10.064 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 18.0.0-6675-gc7471068 (c7471068426370a3754c2250e7729280233d552b) reef (dev)", 2023-10-16T08:39:10.064 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "3", 2023-10-16T08:39:10.065 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2023-10-16T08:39:10.065 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-10-16T08:39:10.065 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "44175", 2023-10-16T08:39:10.065 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2023-10-16T08:39:10.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-16T08:39:10.066 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2023-10-16T08:39:10.066 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2023-10-16T08:39:10.066 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 4594, 2023-10-16T08:39:10.066 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2023-10-16T08:39:10.066 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2023-10-16T08:39:10.067 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2023-10-16T08:39:10.067 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 4594 2023-10-16T08:39:10.067 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-16T08:39:10.067 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-16T08:39:10.067 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-16T08:39:10.067 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.2:0", 2023-10-16T08:39:10.068 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3655138368 2023-10-16T08:39:10.068 INFO:teuthology.orchestra.run.smithi086.stdout: } 2023-10-16T08:39:10.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-16T08:39:10.068 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2023-10-16T08:39:10.068 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2023-10-16T08:39:10.069 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.2:0", 2023-10-16T08:39:10.069 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3655138368 2023-10-16T08:39:10.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2023-10-16T08:39:10.069 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.4594 192.168.0.2:0/3655138368", 2023-10-16T08:39:10.069 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.2:0/3655138368", 2023-10-16T08:39:10.070 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2023-10-16T08:39:10.070 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 6, 2023-10-16T08:39:10.070 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 20, 2023-10-16T08:39:10.070 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2023-10-16T08:39:10.070 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2023-10-16T08:39:10.071 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2023-10-16T08:39:10.071 INFO:teuthology.orchestra.run.smithi086.stdout:} 2023-10-16T08:39:10.077 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:10.077 DEBUG:teuthology.orchestra.run.smithi040:> stat --file-system '--printf=%T 2023-10-16T08:39:10.078 DEBUG:teuthology.orchestra.run.smithi040:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-16T08:39:10.085 INFO:teuthology.orchestra.run.smithi040.stdout:fuseblk 2023-10-16T08:39:10.085 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-16T08:39:10.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:10.086 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-16T08:39:10.143 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:10.144 DEBUG:teuthology.orchestra.run.smithi040:> stat --file-system '--printf=%T 2023-10-16T08:39:10.144 DEBUG:teuthology.orchestra.run.smithi040:> ' -- /home/ubuntu/cephtest/mnt.2 2023-10-16T08:39:10.192 INFO:teuthology.orchestra.run.smithi040.stdout:fuseblk 2023-10-16T08:39:10.193 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-10-16T08:39:10.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:10.193 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-16T08:39:10.253 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:10.254 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2023-10-16T08:39:10.254 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-16T08:39:10.261 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2023-10-16T08:39:10.262 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-16T08:39:10.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:10.262 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-16T08:39:10.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:10.319 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2023-10-16T08:39:10.320 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.3 2023-10-16T08:39:10.367 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2023-10-16T08:39:10.368 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-10-16T08:39:10.368 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:10.369 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-16T08:39:10.423 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-16T08:39:10.443 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:39:10.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.575+0000 7f57fc808640 1 Processor -- start 2023-10-16T08:39:10.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.575+0000 7f57fc808640 1 -- start start 2023-10-16T08:39:10.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.575+0000 7f57fc808640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f57f815d180 0x7f57f806b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:10.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.575+0000 7f57fc808640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f80ad820 0x7f57f80c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:10.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.575+0000 7f57fc808640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57f80cf480 con 0x7f57f815d560 2023-10-16T08:39:10.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.575+0000 7f57fc808640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57f80cf760 con 0x7f57f815d180 2023-10-16T08:39:10.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.575+0000 7f57fc808640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80c8fb0 con 0x7f57f80ad820 2023-10-16T08:39:10.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.575+0000 7f57f659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f57f815d180 0x7f57f806b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:10.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.575+0000 7f57f659c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f57f815d180 0x7f57f806b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48032/0 (socket says 172.21.15.40:48032) 2023-10-16T08:39:10.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.575+0000 7f57f659c640 1 -- 172.21.15.40:0/1829491793 learned_addr learned my addr 172.21.15.40:0/1829491793 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:10.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.575+0000 7f57f5d9b640 1 --1- 172.21.15.40:0/1829491793 >> v1:172.21.15.40:6789/0 conn(0x7f57f815d560 0x7f57f80c9570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54672/0 (socket says 172.21.15.40:54672) 2023-10-16T08:39:10.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.575+0000 7f57f6d9d640 1 --2- 172.21.15.40:0/1829491793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f80ad820 0x7f57f80c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:10.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.575+0000 7f57f659c640 1 -- 172.21.15.40:0/1829491793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f80ad820 msgr2=0x7f57f80c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:10.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.575+0000 7f57f659c640 1 --2- 172.21.15.40:0/1829491793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f80ad820 0x7f57f80c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:10.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.575+0000 7f57f659c640 1 -- 172.21.15.40:0/1829491793 >> v1:172.21.15.40:6789/0 conn(0x7f57f815d560 legacy=0x7f57f80c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:10.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.575+0000 7f57f659c640 1 -- 172.21.15.40:0/1829491793 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f80c9290 con 0x7f57f815d180 2023-10-16T08:39:10.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.575+0000 7f57f559a640 1 -- 172.21.15.40:0/1829491793 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 685264062 0 0) 0x7f57f80cf480 con 0x7f57f815d560 2023-10-16T08:39:10.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57f659c640 1 --2- 172.21.15.40:0/1829491793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f815d180 0x7f57f806b7e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f57f000bb10 tx=0x7f57f001eb80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e355eb7e2e5b47bf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:10.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57f559a640 1 -- 172.21.15.40:0/1829491793 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f00a7c70 con 0x7f57f815d180 2023-10-16T08:39:10.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57f559a640 1 -- 172.21.15.40:0/1829491793 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f00a7dd0 con 0x7f57f815d180 2023-10-16T08:39:10.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57f559a640 1 -- 172.21.15.40:0/1829491793 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f00af660 con 0x7f57f815d180 2023-10-16T08:39:10.586 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57fc808640 1 -- 172.21.15.40:0/1829491793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f815d180 msgr2=0x7f57f806b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:10.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57fc808640 1 --2- 172.21.15.40:0/1829491793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f815d180 0x7f57f806b7e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f57f000bb10 tx=0x7f57f001eb80 comp rx=0 tx=0).stop 2023-10-16T08:39:10.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57fc808640 1 -- 172.21.15.40:0/1829491793 shutdown_connections 2023-10-16T08:39:10.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57fc808640 1 --2- 172.21.15.40:0/1829491793 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f80ad820 0x7f57f80c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:10.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57fc808640 1 --2- 172.21.15.40:0/1829491793 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f815d180 0x7f57f806b7e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:10.587 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57fc808640 1 -- 172.21.15.40:0/1829491793 >> 172.21.15.40:0/1829491793 conn(0x7f57f80ce960 msgr2=0x7f57f80ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:10.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57fc808640 1 -- 172.21.15.40:0/1829491793 shutdown_connections 2023-10-16T08:39:10.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57fc808640 1 -- 172.21.15.40:0/1829491793 wait complete. 2023-10-16T08:39:10.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57fc808640 1 Processor -- start 2023-10-16T08:39:10.588 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57fc808640 1 -- start start 2023-10-16T08:39:10.590 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57fc808640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f80ad820 0x7f57f80b4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:10.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57fc808640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f815d180 0x7f57f80c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:10.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57fc808640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f815d560 0x7f57f80c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:10.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57fc808640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80c7750 con 0x7f57f80ad820 2023-10-16T08:39:10.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57fc808640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80c8a80 con 0x7f57f815d180 2023-10-16T08:39:10.591 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57fc808640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57f80b46d0 con 0x7f57f815d560 2023-10-16T08:39:10.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57f659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f80ad820 0x7f57f80b4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:10.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57f659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f80ad820 0x7f57f80b4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48038/0 (socket says 172.21.15.40:48038) 2023-10-16T08:39:10.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.579+0000 7f57f659c640 1 -- 172.21.15.40:0/792108372 learned_addr learned my addr 172.21.15.40:0/792108372 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:10.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.583+0000 7f57f5d9b640 1 --2- 172.21.15.40:0/792108372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f815d180 0x7f57f80c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:10.592 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.583+0000 7f57f6d9d640 1 --2- 172.21.15.40:0/792108372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f815d560 0x7f57f80c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:10.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.583+0000 7f57f659c640 1 -- 172.21.15.40:0/792108372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f815d560 msgr2=0x7f57f80c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:10.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.583+0000 7f57f659c640 1 --2- 172.21.15.40:0/792108372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f815d560 0x7f57f80c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:10.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.583+0000 7f57f659c640 1 -- 172.21.15.40:0/792108372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f815d180 msgr2=0x7f57f80c7c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:10.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.583+0000 7f57f659c640 1 --2- 172.21.15.40:0/792108372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f815d180 0x7f57f80c7c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:10.593 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.583+0000 7f57f659c640 1 -- 172.21.15.40:0/792108372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f000b770 con 0x7f57f80ad820 2023-10-16T08:39:10.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.583+0000 7f57f659c640 1 --2- 172.21.15.40:0/792108372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f80ad820 0x7f57f80b4190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f57f000b2c0 tx=0x7f57f001b920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:10.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.583+0000 7f57eeffd640 1 -- 172.21.15.40:0/792108372 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f0006ee0 con 0x7f57f80ad820 2023-10-16T08:39:10.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.583+0000 7f57eeffd640 1 -- 172.21.15.40:0/792108372 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57f00afe50 con 0x7f57f80ad820 2023-10-16T08:39:10.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.583+0000 7f57fc808640 1 -- 172.21.15.40:0/792108372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57f80c87a0 con 0x7f57f80ad820 2023-10-16T08:39:10.594 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.583+0000 7f57eeffd640 1 -- 172.21.15.40:0/792108372 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f00b8640 con 0x7f57f80ad820 2023-10-16T08:39:10.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.583+0000 7f57fc808640 1 -- 172.21.15.40:0/792108372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57f80c32b0 con 0x7f57f80ad820 2023-10-16T08:39:10.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.583+0000 7f57fc808640 1 -- 172.21.15.40:0/792108372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57f804e9c0 con 0x7f57f80ad820 2023-10-16T08:39:10.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.583+0000 7f57eeffd640 1 -- 172.21.15.40:0/792108372 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f57f00b6040 con 0x7f57f80ad820 2023-10-16T08:39:10.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.583+0000 7f57eeffd640 1 --2- 172.21.15.40:0/792108372 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f57d80664a0 0x7f57d8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:10.595 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.583+0000 7f57eeffd640 1 -- 172.21.15.40:0/792108372 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f57f0120b10 con 0x7f57f80ad820 2023-10-16T08:39:10.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.583+0000 7f57f5d9b640 1 --2- 172.21.15.40:0/792108372 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f57d80664a0 0x7f57d8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:10.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.583+0000 7f57f5d9b640 1 --2- 172.21.15.40:0/792108372 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f57d80664a0 0x7f57d8068960 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f57f81512d0 tx=0x7f57e0007b60 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:10.596 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.587+0000 7f57eeffd640 1 -- 172.21.15.40:0/792108372 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f57f00ee020 con 0x7f57f80ad820 2023-10-16T08:39:10.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.747+0000 7f57fc808640 1 -- 172.21.15.40:0/792108372 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57f80b48e0 con 0x7f57f80ad820 2023-10-16T08:39:10.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.747+0000 7f57eeffd640 1 -- 172.21.15.40:0/792108372 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f57f00bc090 con 0x7f57f80ad820 2023-10-16T08:39:10.753 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:39:10.754 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:46.611098+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:39:10.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.751+0000 7f57fc808640 1 -- 172.21.15.40:0/792108372 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f57d80664a0 msgr2=0x7f57d8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:10.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.751+0000 7f57fc808640 1 --2- 172.21.15.40:0/792108372 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f57d80664a0 0x7f57d8068960 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f57f81512d0 tx=0x7f57e0007b60 comp rx=0 tx=0).stop 2023-10-16T08:39:10.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.751+0000 7f57fc808640 1 -- 172.21.15.40:0/792108372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f80ad820 msgr2=0x7f57f80b4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:10.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.751+0000 7f57fc808640 1 --2- 172.21.15.40:0/792108372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f80ad820 0x7f57f80b4190 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f57f000b2c0 tx=0x7f57f001b920 comp rx=0 tx=0).stop 2023-10-16T08:39:10.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.751+0000 7f57fc808640 1 -- 172.21.15.40:0/792108372 shutdown_connections 2023-10-16T08:39:10.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.751+0000 7f57fc808640 1 --2- 172.21.15.40:0/792108372 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57f815d560 0x7f57f80c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:10.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.751+0000 7f57fc808640 1 --2- 172.21.15.40:0/792108372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57f815d180 0x7f57f80c7c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:10.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.751+0000 7f57fc808640 1 --2- 172.21.15.40:0/792108372 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f57d80664a0 0x7f57d8068960 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:10.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.751+0000 7f57fc808640 1 --2- 172.21.15.40:0/792108372 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57f80ad820 0x7f57f80b4190 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:10.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.751+0000 7f57fc808640 1 -- 172.21.15.40:0/792108372 >> 172.21.15.40:0/792108372 conn(0x7f57f80ce960 msgr2=0x7f57f80cc650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:10.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.751+0000 7f57fc808640 1 -- 172.21.15.40:0/792108372 shutdown_connections 2023-10-16T08:39:10.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.751+0000 7f57fc808640 1 -- 172.21.15.40:0/792108372 wait complete. 2023-10-16T08:39:10.771 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:38:33.014668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:38:43.055215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:38:43.971666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:39:10.772 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:39:10.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.895+0000 7f10f6911640 1 Processor -- start 2023-10-16T08:39:10.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.895+0000 7f10f6911640 1 -- start start 2023-10-16T08:39:10.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.895+0000 7f10f6911640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f10f00b1c60 0x7f10f01608a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:10.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.895+0000 7f10f6911640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f00b4e40 con 0x7f10f015ceb0 2023-10-16T08:39:10.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.895+0000 7f10f6911640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10f0065a80 con 0x7f10f00aee30 2023-10-16T08:39:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.895+0000 7f10f6911640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10f0066900 con 0x7f10f00b1c60 2023-10-16T08:39:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.895+0000 7f10f4eae640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f10f00b1c60 0x7f10f01608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.895+0000 7f10f4eae640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f10f00b1c60 0x7f10f01608a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48054/0 (socket says 172.21.15.40:48054) 2023-10-16T08:39:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.895+0000 7f10f4eae640 1 -- 172.21.15.40:0/3211338652 learned_addr learned my addr 172.21.15.40:0/3211338652 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:10.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.895+0000 7f10f4eae640 1 -- 172.21.15.40:0/3211338652 >> v1:172.21.15.86:6789/0 conn(0x7f10f00aee30 legacy=0x7f10f00b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:10.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.895+0000 7f10f4eae640 1 -- 172.21.15.40:0/3211338652 >> v1:172.21.15.40:6789/0 conn(0x7f10f015ceb0 legacy=0x7f10f00661f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:10.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.895+0000 7f10f4eae640 1 -- 172.21.15.40:0/3211338652 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10f006de80 con 0x7f10f00b1c60 2023-10-16T08:39:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f4eae640 1 --2- 172.21.15.40:0/3211338652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f00b1c60 0x7f10f01608a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009e50 tx=0x7f10e801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8155df385448315 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:10.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10eeffd640 1 -- 172.21.15.40:0/3211338652 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e80a5810 con 0x7f10f00b1c60 2023-10-16T08:39:10.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10eeffd640 1 -- 172.21.15.40:0/3211338652 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e80a5970 con 0x7f10f00b1c60 2023-10-16T08:39:10.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10eeffd640 1 -- 172.21.15.40:0/3211338652 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e80a5d00 con 0x7f10f00b1c60 2023-10-16T08:39:10.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f6911640 1 -- 172.21.15.40:0/3211338652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f00b1c60 msgr2=0x7f10f01608a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:10.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f6911640 1 --2- 172.21.15.40:0/3211338652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f00b1c60 0x7f10f01608a0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f10e8009e50 tx=0x7f10e801c950 comp rx=0 tx=0).stop 2023-10-16T08:39:10.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f6911640 1 -- 172.21.15.40:0/3211338652 shutdown_connections 2023-10-16T08:39:10.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f6911640 1 --2- 172.21.15.40:0/3211338652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f00b1c60 0x7f10f01608a0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:10.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f6911640 1 -- 172.21.15.40:0/3211338652 >> 172.21.15.40:0/3211338652 conn(0x7f10f00733c0 msgr2=0x7f10f00b2960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:10.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f6911640 1 -- 172.21.15.40:0/3211338652 shutdown_connections 2023-10-16T08:39:10.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f6911640 1 -- 172.21.15.40:0/3211338652 wait complete. 2023-10-16T08:39:10.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f6911640 1 Processor -- start 2023-10-16T08:39:10.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f6911640 1 -- start start 2023-10-16T08:39:10.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f6911640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10f00b1c60 0x7f10f01450d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:10.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f6911640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10f015ceb0 0x7f10f01510b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:10.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f6911640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f014e720 0x7f10f01549f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:10.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f6911640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f015c390 con 0x7f10f014e720 2023-10-16T08:39:10.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f6911640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10f0153fa0 con 0x7f10f00b1c60 2023-10-16T08:39:10.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f6911640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10f015c5a0 con 0x7f10f015ceb0 2023-10-16T08:39:10.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f4eae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f014e720 0x7f10f01549f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:10.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f4eae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f014e720 0x7f10f01549f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48070/0 (socket says 172.21.15.40:48070) 2023-10-16T08:39:10.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f4eae640 1 -- 172.21.15.40:0/1572255527 learned_addr learned my addr 172.21.15.40:0/1572255527 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:10.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10ef7fe640 1 --2- 172.21.15.40:0/1572255527 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10f015ceb0 0x7f10f01510b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:10.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10effff640 1 --2- 172.21.15.40:0/1572255527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10f00b1c60 0x7f10f01450d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:10.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f4eae640 1 -- 172.21.15.40:0/1572255527 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10f015ceb0 msgr2=0x7f10f01510b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:10.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f4eae640 1 --2- 172.21.15.40:0/1572255527 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10f015ceb0 0x7f10f01510b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:10.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f4eae640 1 -- 172.21.15.40:0/1572255527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10f00b1c60 msgr2=0x7f10f01450d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:10.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f4eae640 1 --2- 172.21.15.40:0/1572255527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10f00b1c60 0x7f10f01450d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:10.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.899+0000 7f10f4eae640 1 -- 172.21.15.40:0/1572255527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e8009b30 con 0x7f10f014e720 2023-10-16T08:39:10.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.903+0000 7f10f4eae640 1 --2- 172.21.15.40:0/1572255527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f014e720 0x7f10f01549f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f10e8019560 tx=0x7f10e80196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:10.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.903+0000 7f10ecff9640 1 -- 172.21.15.40:0/1572255527 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e80b8670 con 0x7f10f014e720 2023-10-16T08:39:10.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.903+0000 7f10ecff9640 1 -- 172.21.15.40:0/1572255527 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10e80b87d0 con 0x7f10f014e720 2023-10-16T08:39:10.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.903+0000 7f10f6911640 1 -- 172.21.15.40:0/1572255527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10f01541f0 con 0x7f10f014e720 2023-10-16T08:39:10.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.903+0000 7f10ecff9640 1 -- 172.21.15.40:0/1572255527 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10e80b8b80 con 0x7f10f014e720 2023-10-16T08:39:10.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.903+0000 7f10f6911640 1 -- 172.21.15.40:0/1572255527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10f01515f0 con 0x7f10f014e720 2023-10-16T08:39:10.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.903+0000 7f10f6911640 1 -- 172.21.15.40:0/1572255527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10f004e9c0 con 0x7f10f014e720 2023-10-16T08:39:10.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.903+0000 7f10ecff9640 1 -- 172.21.15.40:0/1572255527 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f10e80b8ce0 con 0x7f10f014e720 2023-10-16T08:39:10.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.903+0000 7f10ecff9640 1 --2- 172.21.15.40:0/1572255527 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f10dc0664a0 0x7f10dc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:10.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.903+0000 7f10ecff9640 1 -- 172.21.15.40:0/1572255527 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10e811f590 con 0x7f10f014e720 2023-10-16T08:39:10.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.903+0000 7f10effff640 1 --2- 172.21.15.40:0/1572255527 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f10dc0664a0 0x7f10dc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:10.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.903+0000 7f10effff640 1 --2- 172.21.15.40:0/1572255527 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f10dc0664a0 0x7f10dc068960 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f10f014f620 tx=0x7f10e0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:10.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:10.907+0000 7f10ecff9640 1 -- 172.21.15.40:0/1572255527 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f10e80ecbf0 con 0x7f10f014e720 2023-10-16T08:39:11.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.071+0000 7f10f6911640 1 -- 172.21.15.40:0/1572255527 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f10f015c6e0 con 0x7f10f014e720 2023-10-16T08:39:11.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.071+0000 7f10ecff9640 1 -- 172.21.15.40:0/1572255527 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f10f015c6e0 con 0x7f10f014e720 2023-10-16T08:39:11.076 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-16T08:39:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.071+0000 7f10f6911640 1 -- 172.21.15.40:0/1572255527 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f10dc0664a0 msgr2=0x7f10dc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.071+0000 7f10f6911640 1 --2- 172.21.15.40:0/1572255527 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f10dc0664a0 0x7f10dc068960 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f10f014f620 tx=0x7f10e0009040 comp rx=0 tx=0).stop 2023-10-16T08:39:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.071+0000 7f10f6911640 1 -- 172.21.15.40:0/1572255527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f014e720 msgr2=0x7f10f01549f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.071+0000 7f10f6911640 1 --2- 172.21.15.40:0/1572255527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f014e720 0x7f10f01549f0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f10e8019560 tx=0x7f10e80196f0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.071+0000 7f10f6911640 1 -- 172.21.15.40:0/1572255527 shutdown_connections 2023-10-16T08:39:11.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.071+0000 7f10f6911640 1 --2- 172.21.15.40:0/1572255527 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f10dc0664a0 0x7f10dc068960 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.071+0000 7f10f6911640 1 --2- 172.21.15.40:0/1572255527 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f10f014e720 0x7f10f01549f0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.071+0000 7f10f6911640 1 --2- 172.21.15.40:0/1572255527 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10f015ceb0 0x7f10f01510b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.071+0000 7f10f6911640 1 --2- 172.21.15.40:0/1572255527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10f00b1c60 0x7f10f01450d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.071+0000 7f10f6911640 1 -- 172.21.15.40:0/1572255527 >> 172.21.15.40:0/1572255527 conn(0x7f10f00733c0 msgr2=0x7f10f00acd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:11.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.071+0000 7f10f6911640 1 -- 172.21.15.40:0/1572255527 shutdown_connections 2023-10-16T08:39:11.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.071+0000 7f10f6911640 1 -- 172.21.15.40:0/1572255527 wait complete. 2023-10-16T08:39:11.095 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:39:11.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8bedefb640 1 Processor -- start 2023-10-16T08:39:11.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8bedefb640 1 -- start start 2023-10-16T08:39:11.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8bedefb640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8be80b8590 0x7f8be80d2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:11.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8bedefb640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8be80d3540 con 0x7f8be80d3140 2023-10-16T08:39:11.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8bedefb640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8be80d1870 con 0x7f8be815cdc0 2023-10-16T08:39:11.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8bedefb640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8be80d1b50 con 0x7f8be80b8590 2023-10-16T08:39:11.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be7fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8be80d3140 0x7f8be80b5da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54690/0 (socket says 172.21.15.40:54690) 2023-10-16T08:39:11.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be7fff640 1 -- 172.21.15.40:0/2608593490 learned_addr learned my addr 172.21.15.40:0/2608593490 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:11.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be6ffd640 1 --2- 172.21.15.40:0/2608593490 >> v2:172.21.15.86:3300/0 conn(0x7f8be80b8590 0x7f8be80d2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:11.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be67fc640 1 -- 172.21.15.40:0/2608593490 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3121514424 0 0) 0x7f8be80d3540 con 0x7f8be80d3140 2023-10-16T08:39:11.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be67fc640 1 -- 172.21.15.40:0/2608593490 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8bd4003530 con 0x7f8be80d3140 2023-10-16T08:39:11.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be67fc640 1 -- 172.21.15.40:0/2608593490 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3406389462 0 0) 0x7f8bd4003530 con 0x7f8be80d3140 2023-10-16T08:39:11.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be67fc640 1 -- 172.21.15.40:0/2608593490 >> v2:172.21.15.86:3300/0 conn(0x7f8be80b8590 msgr2=0x7f8be80d2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:11.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be67fc640 1 --2- 172.21.15.40:0/2608593490 >> v2:172.21.15.86:3300/0 conn(0x7f8be80b8590 0x7f8be80d2c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be67fc640 1 -- 172.21.15.40:0/2608593490 >> v1:172.21.15.86:6789/0 conn(0x7f8be815cdc0 legacy=0x7f8be80d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:11.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be67fc640 1 -- 172.21.15.40:0/2608593490 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be80d1e30 con 0x7f8be80d3140 2023-10-16T08:39:11.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be67fc640 1 -- 172.21.15.40:0/2608593490 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f8be0001680 con 0x7f8be80d3140 2023-10-16T08:39:11.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be67fc640 1 -- 172.21.15.40:0/2608593490 >> v1:172.21.15.40:6789/0 conn(0x7f8be80d3140 legacy=0x7f8be80b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:11.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be67fc640 1 --2- 172.21.15.40:0/2608593490 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8bd40039f0 0x7f8bd4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:11.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be67fc640 1 --2- 172.21.15.40:0/2608593490 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bd40047d0 0x7f8bd4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:11.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be67fc640 1 --2- 172.21.15.40:0/2608593490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd4005190 0x7f8bd400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:11.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be67fc640 1 -- 172.21.15.40:0/2608593490 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be80d1b50 con 0x7f8bd40047d0 2023-10-16T08:39:11.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be67fc640 1 -- 172.21.15.40:0/2608593490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd400c0c0 con 0x7f8bd4005190 2023-10-16T08:39:11.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be67fc640 1 -- 172.21.15.40:0/2608593490 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bd400c280 con 0x7f8bd40039f0 2023-10-16T08:39:11.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be6ffd640 1 --2- 172.21.15.40:0/2608593490 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bd40047d0 0x7f8bd4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:11.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be77fe640 1 --2- 172.21.15.40:0/2608593490 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8bd40039f0 0x7f8bd4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:11.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be6ffd640 1 -- 172.21.15.40:0/2608593490 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8bd40039f0 msgr2=0x7f8bd4003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:11.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be6ffd640 1 --2- 172.21.15.40:0/2608593490 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8bd40039f0 0x7f8bd4003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be6ffd640 1 -- 172.21.15.40:0/2608593490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd4005190 msgr2=0x7f8bd400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:11.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be6ffd640 1 --2- 172.21.15.40:0/2608593490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd4005190 0x7f8bd400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be6ffd640 1 -- 172.21.15.40:0/2608593490 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bd4003530 con 0x7f8bd40047d0 2023-10-16T08:39:11.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be7fff640 1 -- 172.21.15.40:0/2608593490 reap_dead start 2023-10-16T08:39:11.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.223+0000 7f8be77fe640 1 --2- 172.21.15.40:0/2608593490 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8bd40039f0 0x7f8bd4003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:11.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8be6ffd640 1 --2- 172.21.15.40:0/2608593490 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bd40047d0 0x7f8bd4004c50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009ae0 tx=0x7f8bd001ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=341e918cb551f285 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:11.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8be67fc640 1 -- 172.21.15.40:0/2608593490 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd00a5640 con 0x7f8bd40047d0 2023-10-16T08:39:11.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8be67fc640 1 -- 172.21.15.40:0/2608593490 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bd00a57a0 con 0x7f8bd40047d0 2023-10-16T08:39:11.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8be67fc640 1 -- 172.21.15.40:0/2608593490 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bd00a5b50 con 0x7f8bd40047d0 2023-10-16T08:39:11.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8bedefb640 1 -- 172.21.15.40:0/2608593490 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bd40047d0 msgr2=0x7f8bd4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:11.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8bedefb640 1 --2- 172.21.15.40:0/2608593490 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bd40047d0 0x7f8bd4004c50 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8bd0009ae0 tx=0x7f8bd001ca50 comp rx=0 tx=0).stop 2023-10-16T08:39:11.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8bedefb640 1 -- 172.21.15.40:0/2608593490 shutdown_connections 2023-10-16T08:39:11.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8bedefb640 1 --2- 172.21.15.40:0/2608593490 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8bd40047d0 0x7f8bd4004c50 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8bedefb640 1 -- 172.21.15.40:0/2608593490 >> 172.21.15.40:0/2608593490 conn(0x7f8be806a290 msgr2=0x7f8be806a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:11.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8bedefb640 1 -- 172.21.15.40:0/2608593490 shutdown_connections 2023-10-16T08:39:11.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8bedefb640 1 -- 172.21.15.40:0/2608593490 wait complete. 2023-10-16T08:39:11.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8bedefb640 1 Processor -- start 2023-10-16T08:39:11.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8bedefb640 1 -- start start 2023-10-16T08:39:11.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8bedefb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd40047d0 0x7f8be8146860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:11.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8bedefb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8be81f1f70 0x7f8be815e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:11.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8bedefb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8be815e730 0x7f8be80cbed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:11.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8bedefb640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be81548d0 con 0x7f8be815e730 2023-10-16T08:39:11.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8bedefb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8be815ecf0 con 0x7f8bd40047d0 2023-10-16T08:39:11.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8bedefb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8be815efd0 con 0x7f8be81f1f70 2023-10-16T08:39:11.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8be7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8be815e730 0x7f8be80cbed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:11.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8be7fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8be815e730 0x7f8be80cbed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48096/0 (socket says 172.21.15.40:48096) 2023-10-16T08:39:11.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8be7fff640 1 -- 172.21.15.40:0/473520619 learned_addr learned my addr 172.21.15.40:0/473520619 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:11.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8be77fe640 1 --2- 172.21.15.40:0/473520619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd40047d0 0x7f8be8146860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:11.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.227+0000 7f8be6ffd640 1 --2- 172.21.15.40:0/473520619 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8be81f1f70 0x7f8be815e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:11.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.231+0000 7f8be7fff640 1 -- 172.21.15.40:0/473520619 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8be81f1f70 msgr2=0x7f8be815e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:11.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.231+0000 7f8be7fff640 1 --2- 172.21.15.40:0/473520619 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8be81f1f70 0x7f8be815e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.231+0000 7f8be7fff640 1 -- 172.21.15.40:0/473520619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd40047d0 msgr2=0x7f8be8146860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:11.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.231+0000 7f8be7fff640 1 --2- 172.21.15.40:0/473520619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd40047d0 0x7f8be8146860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.231+0000 7f8be7fff640 1 -- 172.21.15.40:0/473520619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8be815f2b0 con 0x7f8be815e730 2023-10-16T08:39:11.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.231+0000 7f8be6ffd640 1 --2- 172.21.15.40:0/473520619 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8be81f1f70 0x7f8be815e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:11.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.231+0000 7f8be7fff640 1 --2- 172.21.15.40:0/473520619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8be815e730 0x7f8be80cbed0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8be0004880 tx=0x7f8be000f570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:11.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.231+0000 7f8bdffff640 1 -- 172.21.15.40:0/473520619 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be0007af0 con 0x7f8be815e730 2023-10-16T08:39:11.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.231+0000 7f8bdffff640 1 -- 172.21.15.40:0/473520619 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be0007c50 con 0x7f8be815e730 2023-10-16T08:39:11.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.231+0000 7f8bedefb640 1 -- 172.21.15.40:0/473520619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be8154b20 con 0x7f8be815e730 2023-10-16T08:39:11.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.231+0000 7f8bdffff640 1 -- 172.21.15.40:0/473520619 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be000a110 con 0x7f8be815e730 2023-10-16T08:39:11.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.231+0000 7f8bedefb640 1 -- 172.21.15.40:0/473520619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8be81f23f0 con 0x7f8be815e730 2023-10-16T08:39:11.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.231+0000 7f8bedefb640 1 -- 172.21.15.40:0/473520619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8be8154d60 con 0x7f8be815e730 2023-10-16T08:39:11.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.231+0000 7f8bdffff640 1 -- 172.21.15.40:0/473520619 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8be000a270 con 0x7f8be815e730 2023-10-16T08:39:11.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.231+0000 7f8bdffff640 1 --2- 172.21.15.40:0/473520619 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f8bc80664d0 0x7f8bc8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:11.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.231+0000 7f8bdffff640 1 -- 172.21.15.40:0/473520619 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8be0086690 con 0x7f8be815e730 2023-10-16T08:39:11.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.231+0000 7f8be77fe640 1 --2- 172.21.15.40:0/473520619 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f8bc80664d0 0x7f8bc8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:11.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.235+0000 7f8be77fe640 1 --2- 172.21.15.40:0/473520619 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f8bc80664d0 0x7f8bc8068990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8000f90 tx=0x7f8bd8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:11.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.235+0000 7f8bdffff640 1 -- 172.21.15.40:0/473520619 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f8be0053d40 con 0x7f8be815e730 2023-10-16T08:39:11.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.399+0000 7f8bedefb640 1 -- 172.21.15.40:0/473520619 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8be8154d60 con 0x7f8be815e730 2023-10-16T08:39:11.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.399+0000 7f8bdffff640 1 -- 172.21.15.40:0/473520619 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8be8154d60 con 0x7f8be815e730 2023-10-16T08:39:11.407 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-16T08:39:11.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.403+0000 7f8bedefb640 1 -- 172.21.15.40:0/473520619 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f8bc80664d0 msgr2=0x7f8bc8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:11.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.403+0000 7f8bedefb640 1 --2- 172.21.15.40:0/473520619 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f8bc80664d0 0x7f8bc8068990 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f8bd8000f90 tx=0x7f8bd8009040 comp rx=0 tx=0).stop 2023-10-16T08:39:11.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.403+0000 7f8bedefb640 1 -- 172.21.15.40:0/473520619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8be815e730 msgr2=0x7f8be80cbed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:11.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.403+0000 7f8bedefb640 1 --2- 172.21.15.40:0/473520619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8be815e730 0x7f8be80cbed0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f8be0004880 tx=0x7f8be000f570 comp rx=0 tx=0).stop 2023-10-16T08:39:11.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.403+0000 7f8bedefb640 1 -- 172.21.15.40:0/473520619 shutdown_connections 2023-10-16T08:39:11.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.403+0000 7f8bedefb640 1 --2- 172.21.15.40:0/473520619 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f8bc80664d0 0x7f8bc8068990 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.403+0000 7f8bedefb640 1 --2- 172.21.15.40:0/473520619 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8be815e730 0x7f8be80cbed0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.403+0000 7f8bedefb640 1 --2- 172.21.15.40:0/473520619 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8be81f1f70 0x7f8be815e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.403+0000 7f8bedefb640 1 --2- 172.21.15.40:0/473520619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8bd40047d0 0x7f8be8146860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.403+0000 7f8bedefb640 1 -- 172.21.15.40:0/473520619 >> 172.21.15.40:0/473520619 conn(0x7f8be806a290 msgr2=0x7f8be8154030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:11.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.403+0000 7f8bedefb640 1 -- 172.21.15.40:0/473520619 shutdown_connections 2023-10-16T08:39:11.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.403+0000 7f8bedefb640 1 -- 172.21.15.40:0/473520619 wait complete. 2023-10-16T08:39:11.425 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:39:11.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.543+0000 7f9886951640 1 Processor -- start 2023-10-16T08:39:11.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.543+0000 7f9886951640 1 -- start start 2023-10-16T08:39:11.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.543+0000 7f9886951640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98800adad0 0x7f98800b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:11.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.543+0000 7f9886951640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f98800adeb0 0x7f98800ca700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:11.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.543+0000 7f9886951640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98800cf3b0 0x7f98800cac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:11.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.543+0000 7f9886951640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98800c98b0 con 0x7f98800cf3b0 2023-10-16T08:39:11.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.543+0000 7f9886951640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98800b47c0 con 0x7f98800adeb0 2023-10-16T08:39:11.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.543+0000 7f9886951640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98800c8f80 con 0x7f98800adad0 2023-10-16T08:39:11.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.543+0000 7f9884eee640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98800cf3b0 0x7f98800cac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:11.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.543+0000 7f9884eee640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98800cf3b0 0x7f98800cac40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48102/0 (socket says 172.21.15.40:48102) 2023-10-16T08:39:11.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.543+0000 7f9884eee640 1 -- 172.21.15.40:0/2415504551 learned_addr learned my addr 172.21.15.40:0/2415504551 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:11.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.543+0000 7f987f7fe640 1 --2- 172.21.15.40:0/2415504551 >> v2:172.21.15.86:3300/0 conn(0x7f98800adeb0 0x7f98800ca700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:11.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.543+0000 7f987ffff640 1 --2- 172.21.15.40:0/2415504551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98800adad0 0x7f98800b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:11.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.543+0000 7f9884eee640 1 -- 172.21.15.40:0/2415504551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98800adad0 msgr2=0x7f98800b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:11.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.543+0000 7f9884eee640 1 --2- 172.21.15.40:0/2415504551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98800adad0 0x7f98800b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.543+0000 7f9884eee640 1 -- 172.21.15.40:0/2415504551 >> v2:172.21.15.86:3300/0 conn(0x7f98800adeb0 msgr2=0x7f98800ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:11.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.543+0000 7f9884eee640 1 --2- 172.21.15.40:0/2415504551 >> v2:172.21.15.86:3300/0 conn(0x7f98800adeb0 0x7f98800ca700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.543+0000 7f9884eee640 1 -- 172.21.15.40:0/2415504551 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98800c9260 con 0x7f98800cf3b0 2023-10-16T08:39:11.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.547+0000 7f9884eee640 1 --2- 172.21.15.40:0/2415504551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98800cf3b0 0x7f98800cac40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9880072610 tx=0x7f987801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a04d8abb046a7cad server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:11.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.547+0000 7f987effd640 1 -- 172.21.15.40:0/2415504551 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98780ac640 con 0x7f98800cf3b0 2023-10-16T08:39:11.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.547+0000 7f987effd640 1 -- 172.21.15.40:0/2415504551 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98780ac7a0 con 0x7f98800cf3b0 2023-10-16T08:39:11.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.547+0000 7f987effd640 1 -- 172.21.15.40:0/2415504551 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98780acb30 con 0x7f98800cf3b0 2023-10-16T08:39:11.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.547+0000 7f9886951640 1 -- 172.21.15.40:0/2415504551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98800cf3b0 msgr2=0x7f98800cac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:11.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.547+0000 7f9886951640 1 --2- 172.21.15.40:0/2415504551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98800cf3b0 0x7f98800cac40 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f9880072610 tx=0x7f987801c440 comp rx=0 tx=0).stop 2023-10-16T08:39:11.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.547+0000 7f9886951640 1 -- 172.21.15.40:0/2415504551 shutdown_connections 2023-10-16T08:39:11.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.547+0000 7f9886951640 1 --2- 172.21.15.40:0/2415504551 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98800cf3b0 0x7f98800cac40 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.547+0000 7f9886951640 1 --2- 172.21.15.40:0/2415504551 >> v2:172.21.15.86:3300/0 conn(0x7f98800adeb0 0x7f98800ca700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.547+0000 7f9886951640 1 --2- 172.21.15.40:0/2415504551 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98800adad0 0x7f98800b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.547+0000 7f9886951640 1 -- 172.21.15.40:0/2415504551 >> 172.21.15.40:0/2415504551 conn(0x7f98800ce890 msgr2=0x7f98800ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:11.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.547+0000 7f9886951640 1 -- 172.21.15.40:0/2415504551 shutdown_connections 2023-10-16T08:39:11.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.547+0000 7f9886951640 1 -- 172.21.15.40:0/2415504551 wait complete. 2023-10-16T08:39:11.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f9886951640 1 Processor -- start 2023-10-16T08:39:11.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f9886951640 1 -- start start 2023-10-16T08:39:11.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f9886951640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98800adad0 0x7f9880152a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:11.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f9886951640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98800adeb0 0x7f9880150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:11.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f9886951640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98800cf3b0 0x7f98800c65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:11.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f9886951640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988015a390 con 0x7f98800adeb0 2023-10-16T08:39:11.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f9886951640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98801508b0 con 0x7f98800cf3b0 2023-10-16T08:39:11.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f9886951640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f988015a0b0 con 0x7f98800adad0 2023-10-16T08:39:11.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f987f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98800adeb0 0x7f9880150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:11.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f987f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98800adeb0 0x7f9880150a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48108/0 (socket says 172.21.15.40:48108) 2023-10-16T08:39:11.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f987f7fe640 1 -- 172.21.15.40:0/2599025436 learned_addr learned my addr 172.21.15.40:0/2599025436 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:11.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f9884eee640 1 --2- 172.21.15.40:0/2599025436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98800cf3b0 0x7f98800c65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:11.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f987ffff640 1 --2- 172.21.15.40:0/2599025436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98800adad0 0x7f9880152a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:11.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f987f7fe640 1 -- 172.21.15.40:0/2599025436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98800adad0 msgr2=0x7f9880152a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:11.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f987f7fe640 1 --2- 172.21.15.40:0/2599025436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98800adad0 0x7f9880152a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f987f7fe640 1 -- 172.21.15.40:0/2599025436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98800cf3b0 msgr2=0x7f98800c65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:11.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f987f7fe640 1 --2- 172.21.15.40:0/2599025436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98800cf3b0 0x7f98800c65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f987f7fe640 1 -- 172.21.15.40:0/2599025436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98780094d0 con 0x7f98800adeb0 2023-10-16T08:39:11.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f987f7fe640 1 --2- 172.21.15.40:0/2599025436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98800adeb0 0x7f9880150a40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f987400d800 tx=0x7f987400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:11.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f987cff9640 1 -- 172.21.15.40:0/2599025436 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987400b6c0 con 0x7f98800adeb0 2023-10-16T08:39:11.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f987cff9640 1 -- 172.21.15.40:0/2599025436 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f987400b820 con 0x7f98800adeb0 2023-10-16T08:39:11.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f9886951640 1 -- 172.21.15.40:0/2599025436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98800c5b00 con 0x7f98800adeb0 2023-10-16T08:39:11.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f987cff9640 1 -- 172.21.15.40:0/2599025436 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f987400bb90 con 0x7f98800adeb0 2023-10-16T08:39:11.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f9886951640 1 -- 172.21.15.40:0/2599025436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98800c60c0 con 0x7f98800adeb0 2023-10-16T08:39:11.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f9886951640 1 -- 172.21.15.40:0/2599025436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98800c6280 con 0x7f98800adeb0 2023-10-16T08:39:11.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f987cff9640 1 -- 172.21.15.40:0/2599025436 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f987400bcf0 con 0x7f98800adeb0 2023-10-16T08:39:11.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.551+0000 7f987cff9640 1 --2- 172.21.15.40:0/2599025436 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f986c0664a0 0x7f986c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:11.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.555+0000 7f987cff9640 1 -- 172.21.15.40:0/2599025436 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9874084440 con 0x7f98800adeb0 2023-10-16T08:39:11.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.555+0000 7f987ffff640 1 --2- 172.21.15.40:0/2599025436 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f986c0664a0 0x7f986c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:11.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.555+0000 7f987ffff640 1 --2- 172.21.15.40:0/2599025436 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f986c0664a0 0x7f986c068960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f988015d9a0 tx=0x7f9870003a20 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:11.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.559+0000 7f987cff9640 1 -- 172.21.15.40:0/2599025436 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f9874051820 con 0x7f98800adeb0 2023-10-16T08:39:11.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.715+0000 7f9886951640 1 -- 172.21.15.40:0/2599025436 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9880067a20 con 0x7f98800adeb0 2023-10-16T08:39:11.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.715+0000 7f987cff9640 1 -- 172.21.15.40:0/2599025436 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9874054e40 con 0x7f98800adeb0 2023-10-16T08:39:11.723 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-16T08:39:11.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.719+0000 7f9886951640 1 -- 172.21.15.40:0/2599025436 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f986c0664a0 msgr2=0x7f986c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:11.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.719+0000 7f9886951640 1 --2- 172.21.15.40:0/2599025436 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f986c0664a0 0x7f986c068960 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f988015d9a0 tx=0x7f9870003a20 comp rx=0 tx=0).stop 2023-10-16T08:39:11.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.719+0000 7f9886951640 1 -- 172.21.15.40:0/2599025436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98800adeb0 msgr2=0x7f9880150a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:11.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.719+0000 7f9886951640 1 --2- 172.21.15.40:0/2599025436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98800adeb0 0x7f9880150a40 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f987400d800 tx=0x7f987400dcc0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.719+0000 7f9886951640 1 -- 172.21.15.40:0/2599025436 shutdown_connections 2023-10-16T08:39:11.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.719+0000 7f9886951640 1 --2- 172.21.15.40:0/2599025436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98800cf3b0 0x7f98800c65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.719+0000 7f9886951640 1 --2- 172.21.15.40:0/2599025436 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f986c0664a0 0x7f986c068960 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.719+0000 7f9886951640 1 --2- 172.21.15.40:0/2599025436 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98800adeb0 0x7f9880150a40 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.719+0000 7f9886951640 1 --2- 172.21.15.40:0/2599025436 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98800adad0 0x7f9880152a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.719+0000 7f9886951640 1 -- 172.21.15.40:0/2599025436 >> 172.21.15.40:0/2599025436 conn(0x7f98800ce890 msgr2=0x7f98800c3a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:11.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.719+0000 7f9886951640 1 -- 172.21.15.40:0/2599025436 shutdown_connections 2023-10-16T08:39:11.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.719+0000 7f9886951640 1 -- 172.21.15.40:0/2599025436 wait complete. 2023-10-16T08:39:11.742 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:39:11.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.855+0000 7f7c72a68640 1 Processor -- start 2023-10-16T08:39:11.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.855+0000 7f7c72a68640 1 -- start start 2023-10-16T08:39:11.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.859+0000 7f7c72a68640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7c6c06bbc0 0x7f7c6c0c3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:11.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.859+0000 7f7c72a68640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c6c161840 con 0x7f7c6c0b10a0 2023-10-16T08:39:11.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.859+0000 7f7c72a68640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c6c0ce4a0 con 0x7f7c6c0c4120 2023-10-16T08:39:11.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.859+0000 7f7c72a68640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c6c0c4590 con 0x7f7c6c06bbc0 2023-10-16T08:39:11.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.859+0000 7f7c6bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7c6c06bbc0 0x7f7c6c0c3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:11.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.859+0000 7f7c6bfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f7c6c06bbc0 0x7f7c6c0c3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48110/0 (socket says 172.21.15.40:48110) 2023-10-16T08:39:11.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.859+0000 7f7c6bfff640 1 -- 172.21.15.40:0/1128112894 learned_addr learned my addr 172.21.15.40:0/1128112894 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:11.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.859+0000 7f7c6bfff640 1 -- 172.21.15.40:0/1128112894 >> v1:172.21.15.86:6789/0 conn(0x7f7c6c0c4120 legacy=0x7f7c6c1608c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:11.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.859+0000 7f7c6bfff640 1 -- 172.21.15.40:0/1128112894 >> v1:172.21.15.40:6789/0 conn(0x7f7c6c0b10a0 legacy=0x7f7c6c069e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:11.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.859+0000 7f7c6b7fe640 1 -- 172.21.15.40:0/1128112894 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1651290243 0 0) 0x7f7c6c161840 con 0x7f7c6c0b10a0 2023-10-16T08:39:11.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.859+0000 7f7c6bfff640 1 -- 172.21.15.40:0/1128112894 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c06a5a0 con 0x7f7c6c06bbc0 2023-10-16T08:39:11.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c6bfff640 1 --2- 172.21.15.40:0/1128112894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c6c06bbc0 0x7f7c6c0c3be0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009870 tx=0x7f7c5401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b72cfa6e7729b886 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:11.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c6b7fe640 1 -- 172.21.15.40:0/1128112894 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c540aa670 con 0x7f7c6c06bbc0 2023-10-16T08:39:11.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c6b7fe640 1 -- 172.21.15.40:0/1128112894 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c540aa7d0 con 0x7f7c6c06bbc0 2023-10-16T08:39:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c6b7fe640 1 -- 172.21.15.40:0/1128112894 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c540aab60 con 0x7f7c6c06bbc0 2023-10-16T08:39:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c72a68640 1 -- 172.21.15.40:0/1128112894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c6c06bbc0 msgr2=0x7f7c6c0c3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c72a68640 1 --2- 172.21.15.40:0/1128112894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c6c06bbc0 0x7f7c6c0c3be0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f7c54009870 tx=0x7f7c5401c510 comp rx=0 tx=0).stop 2023-10-16T08:39:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c72a68640 1 -- 172.21.15.40:0/1128112894 shutdown_connections 2023-10-16T08:39:11.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c72a68640 1 --2- 172.21.15.40:0/1128112894 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c6c06bbc0 0x7f7c6c0c3be0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c72a68640 1 -- 172.21.15.40:0/1128112894 >> 172.21.15.40:0/1128112894 conn(0x7f7c6c099fa0 msgr2=0x7f7c6c0d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:11.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c72a68640 1 -- 172.21.15.40:0/1128112894 shutdown_connections 2023-10-16T08:39:11.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c72a68640 1 -- 172.21.15.40:0/1128112894 wait complete. 2023-10-16T08:39:11.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c72a68640 1 Processor -- start 2023-10-16T08:39:11.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c72a68640 1 -- start start 2023-10-16T08:39:11.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c72a68640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c6c06bbc0 0x7f7c6c1606f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:11.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c72a68640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c6c0b10a0 0x7f7c6c0ae4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:11.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c72a68640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c6c0c4120 0x7f7c6c0aea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:11.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c72a68640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c0afa60 con 0x7f7c6c06bbc0 2023-10-16T08:39:11.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c72a68640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c15fe00 con 0x7f7c6c0b10a0 2023-10-16T08:39:11.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c72a68640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c6c15f1f0 con 0x7f7c6c0c4120 2023-10-16T08:39:11.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c70804640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c6c06bbc0 0x7f7c6c1606f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:11.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c70804640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c6c06bbc0 0x7f7c6c1606f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48122/0 (socket says 172.21.15.40:48122) 2023-10-16T08:39:11.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c70804640 1 -- 172.21.15.40:0/353834129 learned_addr learned my addr 172.21.15.40:0/353834129 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:11.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c6bfff640 1 --2- 172.21.15.40:0/353834129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c6c0b10a0 0x7f7c6c0ae4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:11.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c71005640 1 --2- 172.21.15.40:0/353834129 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c6c0c4120 0x7f7c6c0aea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:11.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c70804640 1 -- 172.21.15.40:0/353834129 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c6c0c4120 msgr2=0x7f7c6c0aea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:11.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c70804640 1 --2- 172.21.15.40:0/353834129 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c6c0c4120 0x7f7c6c0aea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c70804640 1 -- 172.21.15.40:0/353834129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c6c0b10a0 msgr2=0x7f7c6c0ae4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:11.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c70804640 1 --2- 172.21.15.40:0/353834129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c6c0b10a0 0x7f7c6c0ae4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:11.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c70804640 1 -- 172.21.15.40:0/353834129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c540094d0 con 0x7f7c6c06bbc0 2023-10-16T08:39:11.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.863+0000 7f7c70804640 1 --2- 172.21.15.40:0/353834129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c6c06bbc0 0x7f7c6c1606f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c00c410 tx=0x7f7c5c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:11.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.867+0000 7f7c697fa640 1 -- 172.21.15.40:0/353834129 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c0054e0 con 0x7f7c6c06bbc0 2023-10-16T08:39:11.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.867+0000 7f7c697fa640 1 -- 172.21.15.40:0/353834129 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c5c003600 con 0x7f7c6c06bbc0 2023-10-16T08:39:11.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.867+0000 7f7c697fa640 1 -- 172.21.15.40:0/353834129 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c5c0039b0 con 0x7f7c6c06bbc0 2023-10-16T08:39:11.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.867+0000 7f7c72a68640 1 -- 172.21.15.40:0/353834129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c6c15f4d0 con 0x7f7c6c06bbc0 2023-10-16T08:39:11.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.867+0000 7f7c72a68640 1 -- 172.21.15.40:0/353834129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c6c15fa90 con 0x7f7c6c06bbc0 2023-10-16T08:39:11.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.867+0000 7f7c72a68640 1 -- 172.21.15.40:0/353834129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c6c04e9c0 con 0x7f7c6c06bbc0 2023-10-16T08:39:11.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.867+0000 7f7c697fa640 1 -- 172.21.15.40:0/353834129 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f7c5c003b10 con 0x7f7c6c06bbc0 2023-10-16T08:39:11.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.867+0000 7f7c697fa640 1 --2- 172.21.15.40:0/353834129 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7c4c0664a0 0x7f7c4c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:11.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.867+0000 7f7c697fa640 1 -- 172.21.15.40:0/353834129 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7c5c0833a0 con 0x7f7c6c06bbc0 2023-10-16T08:39:11.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.867+0000 7f7c6bfff640 1 --2- 172.21.15.40:0/353834129 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7c4c0664a0 0x7f7c4c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:11.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.867+0000 7f7c6bfff640 1 --2- 172.21.15.40:0/353834129 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7c4c0664a0 0x7f7c4c068960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7c540ae000 tx=0x7f7c540a7000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:11.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:11.871+0000 7f7c697fa640 1 -- 172.21.15.40:0/353834129 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f7c5c054340 con 0x7f7c6c06bbc0 2023-10-16T08:39:12.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.031+0000 7f7c72a68640 1 -- 172.21.15.40:0/353834129 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7c6c06e460 con 0x7f7c6c06bbc0 2023-10-16T08:39:12.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.035+0000 7f7c697fa640 1 -- 172.21.15.40:0/353834129 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f7c5c053ed0 con 0x7f7c6c06bbc0 2023-10-16T08:39:12.039 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:39:12.040 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:46.611098+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:39:12.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.035+0000 7f7c72a68640 1 -- 172.21.15.40:0/353834129 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7c4c0664a0 msgr2=0x7f7c4c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.035+0000 7f7c72a68640 1 --2- 172.21.15.40:0/353834129 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7c4c0664a0 0x7f7c4c068960 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7c540ae000 tx=0x7f7c540a7000 comp rx=0 tx=0).stop 2023-10-16T08:39:12.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.035+0000 7f7c72a68640 1 -- 172.21.15.40:0/353834129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c6c06bbc0 msgr2=0x7f7c6c1606f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.035+0000 7f7c72a68640 1 --2- 172.21.15.40:0/353834129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c6c06bbc0 0x7f7c6c1606f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f7c5c00c410 tx=0x7f7c5c00c8d0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.035+0000 7f7c72a68640 1 -- 172.21.15.40:0/353834129 shutdown_connections 2023-10-16T08:39:12.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.035+0000 7f7c72a68640 1 --2- 172.21.15.40:0/353834129 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f7c6c0c4120 0x7f7c6c0aea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.035+0000 7f7c72a68640 1 --2- 172.21.15.40:0/353834129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c6c0b10a0 0x7f7c6c0ae4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.035+0000 7f7c72a68640 1 --2- 172.21.15.40:0/353834129 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f7c4c0664a0 0x7f7c4c068960 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.035+0000 7f7c72a68640 1 --2- 172.21.15.40:0/353834129 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f7c6c06bbc0 0x7f7c6c1606f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.035+0000 7f7c72a68640 1 -- 172.21.15.40:0/353834129 >> 172.21.15.40:0/353834129 conn(0x7f7c6c099fa0 msgr2=0x7f7c6c06e680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:12.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.035+0000 7f7c72a68640 1 -- 172.21.15.40:0/353834129 shutdown_connections 2023-10-16T08:39:12.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.035+0000 7f7c72a68640 1 -- 172.21.15.40:0/353834129 wait complete. 2023-10-16T08:39:12.059 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:38:33.014668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:38:43.055215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:38:43.971666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:39:12.059 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:39:12.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.187+0000 7f1fcd0f2640 1 Processor -- start 2023-10-16T08:39:12.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.187+0000 7f1fcd0f2640 1 -- start start 2023-10-16T08:39:12.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.187+0000 7f1fcd0f2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fc815cf50 0x7f1fc80ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:12.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.187+0000 7f1fcd0f2640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1fc80ca7a0 0x7f1fc80ccc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:12.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.187+0000 7f1fcd0f2640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fc8071bc0 con 0x7f1fc80cd170 2023-10-16T08:39:12.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.187+0000 7f1fcd0f2640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fc80cd620 con 0x7f1fc80ca7a0 2023-10-16T08:39:12.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.187+0000 7f1fcd0f2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc80ade40 con 0x7f1fc815cf50 2023-10-16T08:39:12.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.187+0000 7f1fc659c640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1fc80ca7a0 0x7f1fc80ccc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:12.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.187+0000 7f1fc6d9d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fc815cf50 0x7f1fc80ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:12.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.187+0000 7f1fc6d9d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fc815cf50 0x7f1fc80ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.40:41938/0 (socket says 172.21.15.40:41938) 2023-10-16T08:39:12.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.187+0000 7f1fc759e640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f1fc80cd170 0x7f1fc80c96b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.40:41586/0 (socket says 172.21.15.40:41586) 2023-10-16T08:39:12.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.187+0000 7f1fc659c640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1fc80ca7a0 0x7f1fc80ccc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.40:60336/0 (socket says 172.21.15.40:60336) 2023-10-16T08:39:12.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.187+0000 7f1fc6d9d640 1 -- 172.21.15.40:0/3500413331 learned_addr learned my addr 172.21.15.40:0/3500413331 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:12.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.187+0000 7f1fc659c640 1 -- 172.21.15.40:0/3500413331 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fc815cf50 msgr2=0x7f1fc80ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:12.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.187+0000 7f1fc659c640 1 --2- 172.21.15.40:0/3500413331 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fc815cf50 0x7f1fc80ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.187+0000 7f1fc659c640 1 -- 172.21.15.40:0/3500413331 >> v1:172.21.15.86:6789/0 conn(0x7f1fc80cd170 legacy=0x7f1fc80c96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.187+0000 7f1fc659c640 1 -- 172.21.15.40:0/3500413331 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fc8070c20 con 0x7f1fc80ca7a0 2023-10-16T08:39:12.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.191+0000 7f1fc659c640 1 --2- 172.21.15.40:0/3500413331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc80ca7a0 0x7f1fc80ccc30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc0097c0 tx=0x7f1fbc01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19e121c82f5db9a8 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:12.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.191+0000 7f1fc5d9b640 1 -- 172.21.15.40:0/3500413331 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc0ab600 con 0x7f1fc80ca7a0 2023-10-16T08:39:12.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.191+0000 7f1fc5d9b640 1 -- 172.21.15.40:0/3500413331 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fbc0a7050 con 0x7f1fc80ca7a0 2023-10-16T08:39:12.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.191+0000 7f1fc5d9b640 1 -- 172.21.15.40:0/3500413331 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fbc0ab980 con 0x7f1fc80ca7a0 2023-10-16T08:39:12.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.191+0000 7f1fcd0f2640 1 -- 172.21.15.40:0/3500413331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc80ca7a0 msgr2=0x7f1fc80ccc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.191+0000 7f1fcd0f2640 1 --2- 172.21.15.40:0/3500413331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc80ca7a0 0x7f1fc80ccc30 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f1fbc0097c0 tx=0x7f1fbc01c450 comp rx=0 tx=0).stop 2023-10-16T08:39:12.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.191+0000 7f1fcd0f2640 1 -- 172.21.15.40:0/3500413331 shutdown_connections 2023-10-16T08:39:12.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.191+0000 7f1fcd0f2640 1 --2- 172.21.15.40:0/3500413331 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc80ca7a0 0x7f1fc80ccc30 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.191+0000 7f1fcd0f2640 1 --2- 172.21.15.40:0/3500413331 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fc815cf50 0x7f1fc80ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.191+0000 7f1fcd0f2640 1 -- 172.21.15.40:0/3500413331 >> 172.21.15.40:0/3500413331 conn(0x7f1fc80cf060 msgr2=0x7f1fc80cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:12.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.191+0000 7f1fcd0f2640 1 -- 172.21.15.40:0/3500413331 shutdown_connections 2023-10-16T08:39:12.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.191+0000 7f1fcd0f2640 1 -- 172.21.15.40:0/3500413331 wait complete. 2023-10-16T08:39:12.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.191+0000 7f1fcd0f2640 1 Processor -- start 2023-10-16T08:39:12.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.191+0000 7f1fcd0f2640 1 -- start start 2023-10-16T08:39:12.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.191+0000 7f1fcd0f2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc80ca7a0 0x7f1fc80c2a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:12.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.191+0000 7f1fcd0f2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fc80cd170 0x7f1fc815b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:12.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.191+0000 7f1fcd0f2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fc815cf50 0x7f1fc8156600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:12.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.191+0000 7f1fcd0f2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc80c88f0 con 0x7f1fc815cf50 2023-10-16T08:39:12.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.191+0000 7f1fcd0f2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc80c66b0 con 0x7f1fc80ca7a0 2023-10-16T08:39:12.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.191+0000 7f1fcd0f2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fc80c6130 con 0x7f1fc80cd170 2023-10-16T08:39:12.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1fc759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fc815cf50 0x7f1fc8156600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:12.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1fc759e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fc815cf50 0x7f1fc8156600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48136/0 (socket says 172.21.15.40:48136) 2023-10-16T08:39:12.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1fc759e640 1 -- 172.21.15.40:0/970586725 learned_addr learned my addr 172.21.15.40:0/970586725 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:12.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1fc6d9d640 1 --2- 172.21.15.40:0/970586725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc80ca7a0 0x7f1fc80c2a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:12.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1fc659c640 1 --2- 172.21.15.40:0/970586725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fc80cd170 0x7f1fc815b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:12.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1fc759e640 1 -- 172.21.15.40:0/970586725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fc80cd170 msgr2=0x7f1fc815b4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1fc759e640 1 --2- 172.21.15.40:0/970586725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fc80cd170 0x7f1fc815b4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1fc759e640 1 -- 172.21.15.40:0/970586725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc80ca7a0 msgr2=0x7f1fc80c2a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1fc759e640 1 --2- 172.21.15.40:0/970586725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc80ca7a0 0x7f1fc80c2a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1fc759e640 1 -- 172.21.15.40:0/970586725 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fbc009420 con 0x7f1fc815cf50 2023-10-16T08:39:12.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1fc6d9d640 1 --2- 172.21.15.40:0/970586725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc80ca7a0 0x7f1fc80c2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:12.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1fc759e640 1 --2- 172.21.15.40:0/970586725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fc815cf50 0x7f1fc8156600 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1fc000c7d0 tx=0x7f1fc000cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:12.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1faf7fe640 1 -- 172.21.15.40:0/970586725 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc000a640 con 0x7f1fc815cf50 2023-10-16T08:39:12.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1faf7fe640 1 -- 172.21.15.40:0/970586725 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fc000a7a0 con 0x7f1fc815cf50 2023-10-16T08:39:12.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1faf7fe640 1 -- 172.21.15.40:0/970586725 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fc000ab10 con 0x7f1fc815cf50 2023-10-16T08:39:12.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1fcd0f2640 1 -- 172.21.15.40:0/970586725 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fc80c57a0 con 0x7f1fc815cf50 2023-10-16T08:39:12.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1fcd0f2640 1 -- 172.21.15.40:0/970586725 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fc80c5450 con 0x7f1fc815cf50 2023-10-16T08:39:12.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1fcd0f2640 1 -- 172.21.15.40:0/970586725 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fc804e9c0 con 0x7f1fc815cf50 2023-10-16T08:39:12.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1faf7fe640 1 -- 172.21.15.40:0/970586725 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1fc000ac70 con 0x7f1fc815cf50 2023-10-16T08:39:12.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1faf7fe640 1 --2- 172.21.15.40:0/970586725 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1fb40664a0 0x7f1fb4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:12.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1faf7fe640 1 -- 172.21.15.40:0/970586725 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1fc0083380 con 0x7f1fc815cf50 2023-10-16T08:39:12.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1fc6d9d640 1 --2- 172.21.15.40:0/970586725 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1fb40664a0 0x7f1fb4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:12.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.195+0000 7f1fc6d9d640 1 --2- 172.21.15.40:0/970586725 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1fb40664a0 0x7f1fb4068960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1fc815c1a0 tx=0x7f1fb8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:12.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.199+0000 7f1faf7fe640 1 -- 172.21.15.40:0/970586725 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f1fc0050940 con 0x7f1fc815cf50 2023-10-16T08:39:12.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.355+0000 7f1fcd0f2640 1 -- 172.21.15.40:0/970586725 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1fc8071bc0 con 0x7f1fc815cf50 2023-10-16T08:39:12.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.359+0000 7f1faf7fe640 1 -- 172.21.15.40:0/970586725 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1fc0017020 con 0x7f1fc815cf50 2023-10-16T08:39:12.363 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-16T08:39:12.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.359+0000 7f1fcd0f2640 1 -- 172.21.15.40:0/970586725 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1fb40664a0 msgr2=0x7f1fb4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.359+0000 7f1fcd0f2640 1 --2- 172.21.15.40:0/970586725 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1fb40664a0 0x7f1fb4068960 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1fc815c1a0 tx=0x7f1fb8008040 comp rx=0 tx=0).stop 2023-10-16T08:39:12.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.359+0000 7f1fcd0f2640 1 -- 172.21.15.40:0/970586725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fc815cf50 msgr2=0x7f1fc8156600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.359+0000 7f1fcd0f2640 1 --2- 172.21.15.40:0/970586725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fc815cf50 0x7f1fc8156600 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1fc000c7d0 tx=0x7f1fc000cc90 comp rx=0 tx=0).stop 2023-10-16T08:39:12.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.359+0000 7f1fcd0f2640 1 -- 172.21.15.40:0/970586725 shutdown_connections 2023-10-16T08:39:12.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.359+0000 7f1fcd0f2640 1 --2- 172.21.15.40:0/970586725 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1fb40664a0 0x7f1fb4068960 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.359+0000 7f1fcd0f2640 1 --2- 172.21.15.40:0/970586725 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1fc815cf50 0x7f1fc8156600 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.359+0000 7f1fcd0f2640 1 --2- 172.21.15.40:0/970586725 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1fc80cd170 0x7f1fc815b4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.359+0000 7f1fcd0f2640 1 --2- 172.21.15.40:0/970586725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fc80ca7a0 0x7f1fc80c2a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.359+0000 7f1fcd0f2640 1 -- 172.21.15.40:0/970586725 >> 172.21.15.40:0/970586725 conn(0x7f1fc80cf060 msgr2=0x7f1fc81590c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:12.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.359+0000 7f1fcd0f2640 1 -- 172.21.15.40:0/970586725 shutdown_connections 2023-10-16T08:39:12.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.359+0000 7f1fcd0f2640 1 -- 172.21.15.40:0/970586725 wait complete. 2023-10-16T08:39:12.381 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:39:12.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.491+0000 7f6f6e54e640 1 Processor -- start 2023-10-16T08:39:12.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.491+0000 7f6f6e54e640 1 -- start start 2023-10-16T08:39:12.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.491+0000 7f6f6e54e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6f6815cea0 0x7f6f681609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:12.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.491+0000 7f6f6e54e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6f680b8550 0x7f6f680cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:12.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.491+0000 7f6f6e54e640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f680d11c0 con 0x7f6f680cde60 2023-10-16T08:39:12.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.491+0000 7f6f6e54e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f680ce310 con 0x7f6f680b8550 2023-10-16T08:39:12.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.491+0000 7f6f6e54e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f681618f0 con 0x7f6f6815cea0 2023-10-16T08:39:12.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.491+0000 7f6f677fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6f680b8550 0x7f6f680cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:12.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.491+0000 7f6f677fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f6f680b8550 0x7f6f680cd920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48142/0 (socket says 172.21.15.40:48142) 2023-10-16T08:39:12.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.491+0000 7f6f677fe640 1 -- 172.21.15.40:0/1154725528 learned_addr learned my addr 172.21.15.40:0/1154725528 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:12.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.491+0000 7f6f67fff640 1 --2- 172.21.15.40:0/1154725528 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6f6815cea0 0x7f6f681609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:12.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.491+0000 7f6f677fe640 1 -- 172.21.15.40:0/1154725528 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6f6815cea0 msgr2=0x7f6f681609c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:12.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.491+0000 7f6f677fe640 1 --2- 172.21.15.40:0/1154725528 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6f6815cea0 0x7f6f681609c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.491+0000 7f6f677fe640 1 -- 172.21.15.40:0/1154725528 >> v1:172.21.15.86:6789/0 conn(0x7f6f680cde60 legacy=0x7f6f680b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.491+0000 7f6f677fe640 1 -- 172.21.15.40:0/1154725528 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f6815d280 con 0x7f6f680b8550 2023-10-16T08:39:12.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f677fe640 1 --2- 172.21.15.40:0/1154725528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f680b8550 0x7f6f680cd920 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009870 tx=0x7f6f5c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537ddba689c90dde server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:12.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f66ffd640 1 -- 172.21.15.40:0/1154725528 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c0a7070 con 0x7f6f680b8550 2023-10-16T08:39:12.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f66ffd640 1 -- 172.21.15.40:0/1154725528 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5c0ab430 con 0x7f6f680b8550 2023-10-16T08:39:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f66ffd640 1 -- 172.21.15.40:0/1154725528 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5c0ab7c0 con 0x7f6f680b8550 2023-10-16T08:39:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f6e54e640 1 -- 172.21.15.40:0/1154725528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f680b8550 msgr2=0x7f6f680cd920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f6e54e640 1 --2- 172.21.15.40:0/1154725528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f680b8550 0x7f6f680cd920 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c009870 tx=0x7f6f5c01c510 comp rx=0 tx=0).stop 2023-10-16T08:39:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f6e54e640 1 -- 172.21.15.40:0/1154725528 shutdown_connections 2023-10-16T08:39:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f6e54e640 1 --2- 172.21.15.40:0/1154725528 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f680b8550 0x7f6f680cd920 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f6e54e640 1 --2- 172.21.15.40:0/1154725528 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6f6815cea0 0x7f6f681609c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f6e54e640 1 -- 172.21.15.40:0/1154725528 >> 172.21.15.40:0/1154725528 conn(0x7f6f68065a80 msgr2=0x7f6f68065e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:12.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f6e54e640 1 -- 172.21.15.40:0/1154725528 shutdown_connections 2023-10-16T08:39:12.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f6e54e640 1 -- 172.21.15.40:0/1154725528 wait complete. 2023-10-16T08:39:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f6e54e640 1 Processor -- start 2023-10-16T08:39:12.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f6e54e640 1 -- start start 2023-10-16T08:39:12.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f6e54e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f680cde60 0x7f6f680c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:12.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f6e54e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f6815cea0 0x7f6f680cbba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:12.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f6e54e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6f681483a0 0x7f6f680b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:12.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f6e54e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f680c79b0 con 0x7f6f680cde60 2023-10-16T08:39:12.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f6e54e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f680c9b90 con 0x7f6f6815cea0 2023-10-16T08:39:12.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f6e54e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f680c9460 con 0x7f6f681483a0 2023-10-16T08:39:12.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f67fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f680cde60 0x7f6f680c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:12.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f67fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f680cde60 0x7f6f680c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48152/0 (socket says 172.21.15.40:48152) 2023-10-16T08:39:12.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f67fff640 1 -- 172.21.15.40:0/209675870 learned_addr learned my addr 172.21.15.40:0/209675870 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:12.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f6caeb640 1 --2- 172.21.15.40:0/209675870 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6f681483a0 0x7f6f680b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:12.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.495+0000 7f6f677fe640 1 --2- 172.21.15.40:0/209675870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f6815cea0 0x7f6f680cbba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:12.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.499+0000 7f6f67fff640 1 -- 172.21.15.40:0/209675870 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6f681483a0 msgr2=0x7f6f680b3e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.499+0000 7f6f67fff640 1 --2- 172.21.15.40:0/209675870 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6f681483a0 0x7f6f680b3e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.499+0000 7f6f67fff640 1 -- 172.21.15.40:0/209675870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f6815cea0 msgr2=0x7f6f680cbba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.499+0000 7f6f67fff640 1 --2- 172.21.15.40:0/209675870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f6815cea0 0x7f6f680cbba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.499+0000 7f6f67fff640 1 -- 172.21.15.40:0/209675870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f5c0094d0 con 0x7f6f680cde60 2023-10-16T08:39:12.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.499+0000 7f6f67fff640 1 --2- 172.21.15.40:0/209675870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f680cde60 0x7f6f680c8dc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800d830 tx=0x7f6f5800dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:12.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.499+0000 7f6f64ff9640 1 -- 172.21.15.40:0/209675870 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800b6c0 con 0x7f6f680cde60 2023-10-16T08:39:12.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.499+0000 7f6f64ff9640 1 -- 172.21.15.40:0/209675870 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f5800b860 con 0x7f6f680cde60 2023-10-16T08:39:12.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.499+0000 7f6f6e54e640 1 -- 172.21.15.40:0/209675870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f680c7110 con 0x7f6f680cde60 2023-10-16T08:39:12.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.499+0000 7f6f64ff9640 1 -- 172.21.15.40:0/209675870 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f5800bc10 con 0x7f6f680cde60 2023-10-16T08:39:12.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.499+0000 7f6f6e54e640 1 -- 172.21.15.40:0/209675870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f680c76d0 con 0x7f6f680cde60 2023-10-16T08:39:12.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.499+0000 7f6f6e54e640 1 -- 172.21.15.40:0/209675870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f6804e9c0 con 0x7f6f680cde60 2023-10-16T08:39:12.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.499+0000 7f6f64ff9640 1 -- 172.21.15.40:0/209675870 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6f5800bd70 con 0x7f6f680cde60 2023-10-16T08:39:12.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.499+0000 7f6f64ff9640 1 --2- 172.21.15.40:0/209675870 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f6f540664a0 0x7f6f54068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:12.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.499+0000 7f6f64ff9640 1 -- 172.21.15.40:0/209675870 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6f58084460 con 0x7f6f680cde60 2023-10-16T08:39:12.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.499+0000 7f6f677fe640 1 --2- 172.21.15.40:0/209675870 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f6f540664a0 0x7f6f54068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:12.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.499+0000 7f6f677fe640 1 --2- 172.21.15.40:0/209675870 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f6f540664a0 0x7f6f54068960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0af000 tx=0x7f6f5c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:12.517 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.503+0000 7f6f64ff9640 1 -- 172.21.15.40:0/209675870 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f6f580518e0 con 0x7f6f680cde60 2023-10-16T08:39:12.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.651+0000 7f6f6e54e640 1 -- 172.21.15.40:0/209675870 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6f680d08d0 con 0x7f6f680cde60 2023-10-16T08:39:12.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.651+0000 7f6f64ff9640 1 -- 172.21.15.40:0/209675870 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6f58054f00 con 0x7f6f680cde60 2023-10-16T08:39:12.659 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-16T08:39:12.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.655+0000 7f6f6e54e640 1 -- 172.21.15.40:0/209675870 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f6f540664a0 msgr2=0x7f6f54068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.655+0000 7f6f6e54e640 1 --2- 172.21.15.40:0/209675870 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f6f540664a0 0x7f6f54068960 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f6f5c0af000 tx=0x7f6f5c008040 comp rx=0 tx=0).stop 2023-10-16T08:39:12.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.655+0000 7f6f6e54e640 1 -- 172.21.15.40:0/209675870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f680cde60 msgr2=0x7f6f680c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.655+0000 7f6f6e54e640 1 --2- 172.21.15.40:0/209675870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f680cde60 0x7f6f680c8dc0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f6f5800d830 tx=0x7f6f5800dcf0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.655+0000 7f6f6e54e640 1 -- 172.21.15.40:0/209675870 shutdown_connections 2023-10-16T08:39:12.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.655+0000 7f6f6e54e640 1 --2- 172.21.15.40:0/209675870 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6f681483a0 0x7f6f680b3e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.655+0000 7f6f6e54e640 1 --2- 172.21.15.40:0/209675870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f6815cea0 0x7f6f680cbba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.655+0000 7f6f6e54e640 1 --2- 172.21.15.40:0/209675870 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f6f540664a0 0x7f6f54068960 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.655+0000 7f6f6e54e640 1 --2- 172.21.15.40:0/209675870 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6f680cde60 0x7f6f680c8dc0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.655+0000 7f6f6e54e640 1 -- 172.21.15.40:0/209675870 >> 172.21.15.40:0/209675870 conn(0x7f6f68065a80 msgr2=0x7f6f680d2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:12.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.655+0000 7f6f6e54e640 1 -- 172.21.15.40:0/209675870 shutdown_connections 2023-10-16T08:39:12.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.655+0000 7f6f6e54e640 1 -- 172.21.15.40:0/209675870 wait complete. 2023-10-16T08:39:12.676 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:39:12.802 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.795+0000 7f87a1bbc640 1 Processor -- start 2023-10-16T08:39:12.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.795+0000 7f87a1bbc640 1 -- start start 2023-10-16T08:39:12.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.799+0000 7f87a1bbc640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f879c0adad0 0x7f879c0b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:12.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.799+0000 7f87a1bbc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f879c0c98b0 con 0x7f879c0adeb0 2023-10-16T08:39:12.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.799+0000 7f87a1bbc640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f879c0b47c0 con 0x7f879c0cae10 2023-10-16T08:39:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.799+0000 7f87a1bbc640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f879c0c8f80 con 0x7f879c0adad0 2023-10-16T08:39:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.799+0000 7f879affd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f879c0adeb0 0x7f879c0ca700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54710/0 (socket says 172.21.15.40:54710) 2023-10-16T08:39:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.799+0000 7f879affd640 1 -- 172.21.15.40:0/3903031174 learned_addr learned my addr 172.21.15.40:0/3903031174 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:12.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.799+0000 7f879b7fe640 1 --2- 172.21.15.40:0/3903031174 >> v2:172.21.15.86:3300/0 conn(0x7f879c0adad0 0x7f879c0b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.799+0000 7f879a7fc640 1 -- 172.21.15.40:0/3903031174 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 227578655 0 0) 0x7f879c0c98b0 con 0x7f879c0adeb0 2023-10-16T08:39:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.799+0000 7f879a7fc640 1 -- 172.21.15.40:0/3903031174 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8784003530 con 0x7f879c0adeb0 2023-10-16T08:39:12.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.799+0000 7f879b7fe640 1 -- 172.21.15.40:0/3903031174 >> v1:172.21.15.86:6789/0 conn(0x7f879c0cae10 legacy=0x7f879c15a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.799+0000 7f879b7fe640 1 -- 172.21.15.40:0/3903031174 >> v1:172.21.15.40:6789/0 conn(0x7f879c0adeb0 legacy=0x7f879c0ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.799+0000 7f879b7fe640 1 -- 172.21.15.40:0/3903031174 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f879c0c9260 con 0x7f879c0adad0 2023-10-16T08:39:12.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.799+0000 7f879b7fe640 1 --2- 172.21.15.40:0/3903031174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f879c0adad0 0x7f879c0b41e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f878c0097c0 tx=0x7f878c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3e19932689b24146 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:12.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.799+0000 7f879a7fc640 1 -- 172.21.15.40:0/3903031174 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c0ab670 con 0x7f879c0adad0 2023-10-16T08:39:12.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f879a7fc640 1 -- 172.21.15.40:0/3903031174 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f878c0ab7d0 con 0x7f879c0adad0 2023-10-16T08:39:12.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f879a7fc640 1 -- 172.21.15.40:0/3903031174 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f878c0abb40 con 0x7f879c0adad0 2023-10-16T08:39:12.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f87a1bbc640 1 -- 172.21.15.40:0/3903031174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f879c0adad0 msgr2=0x7f879c0b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f87a1bbc640 1 --2- 172.21.15.40:0/3903031174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f879c0adad0 0x7f879c0b41e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f878c0097c0 tx=0x7f878c01c450 comp rx=0 tx=0).stop 2023-10-16T08:39:12.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f87a1bbc640 1 -- 172.21.15.40:0/3903031174 shutdown_connections 2023-10-16T08:39:12.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f87a1bbc640 1 --2- 172.21.15.40:0/3903031174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f879c0adad0 0x7f879c0b41e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f87a1bbc640 1 -- 172.21.15.40:0/3903031174 >> 172.21.15.40:0/3903031174 conn(0x7f879c0ce890 msgr2=0x7f879c0ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:12.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f87a1bbc640 1 -- 172.21.15.40:0/3903031174 shutdown_connections 2023-10-16T08:39:12.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f87a1bbc640 1 -- 172.21.15.40:0/3903031174 wait complete. 2023-10-16T08:39:12.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f87a1bbc640 1 Processor -- start 2023-10-16T08:39:12.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f87a1bbc640 1 -- start start 2023-10-16T08:39:12.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f87a1bbc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f879c0adad0 0x7f879c13cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:12.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f87a1bbc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f879c0adeb0 0x7f879c0cc460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:12.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f87a1bbc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879c0cae10 0x7f879c159300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:12.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f87a1bbc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c14c500 con 0x7f879c0cae10 2023-10-16T08:39:12.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f87a1bbc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f879c0cc9a0 con 0x7f879c0adeb0 2023-10-16T08:39:12.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f87a1bbc640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f879c0c8bb0 con 0x7f879c0adad0 2023-10-16T08:39:12.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f879bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879c0cae10 0x7f879c159300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:12.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f879bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879c0cae10 0x7f879c159300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48164/0 (socket says 172.21.15.40:48164) 2023-10-16T08:39:12.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f879bfff640 1 -- 172.21.15.40:0/1600106196 learned_addr learned my addr 172.21.15.40:0/1600106196 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:12.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f879b7fe640 1 --2- 172.21.15.40:0/1600106196 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f879c0adad0 0x7f879c13cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f879affd640 1 --2- 172.21.15.40:0/1600106196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f879c0adeb0 0x7f879c0cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f879bfff640 1 -- 172.21.15.40:0/1600106196 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f879c0adad0 msgr2=0x7f879c13cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f879bfff640 1 --2- 172.21.15.40:0/1600106196 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f879c0adad0 0x7f879c13cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f879bfff640 1 -- 172.21.15.40:0/1600106196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f879c0adeb0 msgr2=0x7f879c0cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f879bfff640 1 --2- 172.21.15.40:0/1600106196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f879c0adeb0 0x7f879c0cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f879bfff640 1 -- 172.21.15.40:0/1600106196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f878c009420 con 0x7f879c0cae10 2023-10-16T08:39:12.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f879affd640 1 --2- 172.21.15.40:0/1600106196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f879c0adeb0 0x7f879c0cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:12.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f879bfff640 1 --2- 172.21.15.40:0/1600106196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879c0cae10 0x7f879c159300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f879400c730 tx=0x7f879400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:12.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f8783fff640 1 -- 172.21.15.40:0/1600106196 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8794006d60 con 0x7f879c0cae10 2023-10-16T08:39:12.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f8783fff640 1 -- 172.21.15.40:0/1600106196 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8794003980 con 0x7f879c0cae10 2023-10-16T08:39:12.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f87a1bbc640 1 -- 172.21.15.40:0/1600106196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f879c14f200 con 0x7f879c0cae10 2023-10-16T08:39:12.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f8783fff640 1 -- 172.21.15.40:0/1600106196 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8794003d30 con 0x7f879c0cae10 2023-10-16T08:39:12.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.803+0000 7f87a1bbc640 1 -- 172.21.15.40:0/1600106196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f879c14c120 con 0x7f879c0cae10 2023-10-16T08:39:12.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.807+0000 7f87a1bbc640 1 -- 172.21.15.40:0/1600106196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f879c067a20 con 0x7f879c0cae10 2023-10-16T08:39:12.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.807+0000 7f8783fff640 1 -- 172.21.15.40:0/1600106196 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f87940050d0 con 0x7f879c0cae10 2023-10-16T08:39:12.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.807+0000 7f8783fff640 1 --2- 172.21.15.40:0/1600106196 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f87880664a0 0x7f8788068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:12.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.807+0000 7f8783fff640 1 -- 172.21.15.40:0/1600106196 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f87940838b0 con 0x7f879c0cae10 2023-10-16T08:39:12.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.807+0000 7f879b7fe640 1 --2- 172.21.15.40:0/1600106196 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f87880664a0 0x7f8788068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:12.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.807+0000 7f879b7fe640 1 --2- 172.21.15.40:0/1600106196 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f87880664a0 0x7f8788068960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f878c0af000 tx=0x7f878c0a7000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:12.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.811+0000 7f8783fff640 1 -- 172.21.15.40:0/1600106196 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f8794050d30 con 0x7f879c0cae10 2023-10-16T08:39:12.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.963+0000 7f87a1bbc640 1 -- 172.21.15.40:0/1600106196 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f879c04e9c0 con 0x7f879c0cae10 2023-10-16T08:39:12.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.963+0000 7f8783fff640 1 -- 172.21.15.40:0/1600106196 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8794054350 con 0x7f879c0cae10 2023-10-16T08:39:12.971 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-16T08:39:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.967+0000 7f87a1bbc640 1 -- 172.21.15.40:0/1600106196 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f87880664a0 msgr2=0x7f8788068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.967+0000 7f87a1bbc640 1 --2- 172.21.15.40:0/1600106196 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f87880664a0 0x7f8788068960 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f878c0af000 tx=0x7f878c0a7000 comp rx=0 tx=0).stop 2023-10-16T08:39:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.967+0000 7f87a1bbc640 1 -- 172.21.15.40:0/1600106196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879c0cae10 msgr2=0x7f879c159300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:12.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.967+0000 7f87a1bbc640 1 --2- 172.21.15.40:0/1600106196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879c0cae10 0x7f879c159300 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f879400c730 tx=0x7f879400cbf0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.967+0000 7f87a1bbc640 1 -- 172.21.15.40:0/1600106196 shutdown_connections 2023-10-16T08:39:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.967+0000 7f87a1bbc640 1 --2- 172.21.15.40:0/1600106196 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f87880664a0 0x7f8788068960 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.967+0000 7f87a1bbc640 1 --2- 172.21.15.40:0/1600106196 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f879c0cae10 0x7f879c159300 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.967+0000 7f87a1bbc640 1 --2- 172.21.15.40:0/1600106196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f879c0adeb0 0x7f879c0cc460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.967+0000 7f87a1bbc640 1 --2- 172.21.15.40:0/1600106196 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f879c0adad0 0x7f879c13cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.967+0000 7f87a1bbc640 1 -- 172.21.15.40:0/1600106196 >> 172.21.15.40:0/1600106196 conn(0x7f879c0ce890 msgr2=0x7f879c157c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.967+0000 7f87a1bbc640 1 -- 172.21.15.40:0/1600106196 shutdown_connections 2023-10-16T08:39:12.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:12.967+0000 7f87a1bbc640 1 -- 172.21.15.40:0/1600106196 wait complete. 2023-10-16T08:39:12.989 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:39:13.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa53d497640 1 Processor -- start 2023-10-16T08:39:13.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa53d497640 1 -- start start 2023-10-16T08:39:13.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa53d497640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5380ad820 0x7fa53806b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:13.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa53d497640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5380cf480 con 0x7fa53815d180 2023-10-16T08:39:13.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa53d497640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5380cf760 con 0x7fa53815d560 2023-10-16T08:39:13.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa53d497640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5380c8fb0 con 0x7fa5380ad820 2023-10-16T08:39:13.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa536ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa53815d180 0x7fa5380c9570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:54720/0 (socket says 172.21.15.40:54720) 2023-10-16T08:39:13.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa536ffd640 1 -- 172.21.15.40:0/3644709043 learned_addr learned my addr 172.21.15.40:0/3644709043 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:13.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa5377fe640 1 --2- 172.21.15.40:0/3644709043 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5380ad820 0x7fa53806b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:13.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa535ffb640 1 -- 172.21.15.40:0/3644709043 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3890123254 0 0) 0x7fa5380cf480 con 0x7fa53815d180 2023-10-16T08:39:13.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa535ffb640 1 -- 172.21.15.40:0/3644709043 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa520005740 con 0x7fa53815d180 2023-10-16T08:39:13.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa535ffb640 1 -- 172.21.15.40:0/3644709043 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3213988275 0 0) 0x7fa520005740 con 0x7fa53815d180 2023-10-16T08:39:13.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa535ffb640 1 -- 172.21.15.40:0/3644709043 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5380ad820 msgr2=0x7fa53806b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:13.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa535ffb640 1 --2- 172.21.15.40:0/3644709043 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5380ad820 0x7fa53806b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa535ffb640 1 -- 172.21.15.40:0/3644709043 >> v1:172.21.15.86:6789/0 conn(0x7fa53815d560 legacy=0x7fa5380cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:13.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa535ffb640 1 -- 172.21.15.40:0/3644709043 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5380c9290 con 0x7fa53815d180 2023-10-16T08:39:13.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa535ffb640 1 -- 172.21.15.40:0/3644709043 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7fa528001680 con 0x7fa53815d180 2023-10-16T08:39:13.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa535ffb640 1 -- 172.21.15.40:0/3644709043 >> v1:172.21.15.40:6789/0 conn(0x7fa53815d180 legacy=0x7fa5380c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:13.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa535ffb640 1 --2- 172.21.15.40:0/3644709043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa520005c30 0x7fa520006050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:13.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa535ffb640 1 --2- 172.21.15.40:0/3644709043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5200069e0 0x7fa520006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:13.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa535ffb640 1 --2- 172.21.15.40:0/3644709043 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5380ad820 0x7fa520007450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:13.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa535ffb640 1 -- 172.21.15.40:0/3644709043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5380c8fb0 con 0x7fa520005c30 2023-10-16T08:39:13.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa535ffb640 1 -- 172.21.15.40:0/3644709043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa52000dd60 con 0x7fa5200069e0 2023-10-16T08:39:13.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa535ffb640 1 -- 172.21.15.40:0/3644709043 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa52000df20 con 0x7fa5380ad820 2023-10-16T08:39:13.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa536ffd640 1 --2- 172.21.15.40:0/3644709043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa520005c30 0x7fa520006050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:13.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa5367fc640 1 --2- 172.21.15.40:0/3644709043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5200069e0 0x7fa520006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:13.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa5377fe640 1 --2- 172.21.15.40:0/3644709043 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5380ad820 0x7fa520007450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:13.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa536ffd640 1 -- 172.21.15.40:0/3644709043 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5380ad820 msgr2=0x7fa520007450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:13.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa536ffd640 1 --2- 172.21.15.40:0/3644709043 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5380ad820 0x7fa520007450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa536ffd640 1 -- 172.21.15.40:0/3644709043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5200069e0 msgr2=0x7fa520006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:13.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa536ffd640 1 --2- 172.21.15.40:0/3644709043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa5200069e0 0x7fa520006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.111+0000 7fa536ffd640 1 -- 172.21.15.40:0/3644709043 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa520005740 con 0x7fa520005c30 2023-10-16T08:39:13.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.115+0000 7fa536ffd640 1 --2- 172.21.15.40:0/3644709043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa520005c30 0x7fa520006050 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa52800c210 tx=0x7fa52801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fff9f960ec4e50 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:13.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.115+0000 7fa535ffb640 1 -- 172.21.15.40:0/3644709043 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa528005ae0 con 0x7fa520005c30 2023-10-16T08:39:13.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.115+0000 7fa535ffb640 1 -- 172.21.15.40:0/3644709043 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa528005c40 con 0x7fa520005c30 2023-10-16T08:39:13.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.115+0000 7fa535ffb640 1 -- 172.21.15.40:0/3644709043 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa528007070 con 0x7fa520005c30 2023-10-16T08:39:13.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.115+0000 7fa53d497640 1 -- 172.21.15.40:0/3644709043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa520005c30 msgr2=0x7fa520006050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:13.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.115+0000 7fa53d497640 1 --2- 172.21.15.40:0/3644709043 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa520005c30 0x7fa520006050 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa52800c210 tx=0x7fa52801cc20 comp rx=0 tx=0).stop 2023-10-16T08:39:13.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.115+0000 7fa5377fe640 1 -- 172.21.15.40:0/3644709043 reap_dead start 2023-10-16T08:39:13.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.115+0000 7fa53d497640 1 -- 172.21.15.40:0/3644709043 shutdown_connections 2023-10-16T08:39:13.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.115+0000 7fa53d497640 1 -- 172.21.15.40:0/3644709043 >> 172.21.15.40:0/3644709043 conn(0x7fa5380ce960 msgr2=0x7fa5380ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:13.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.115+0000 7fa53d497640 1 -- 172.21.15.40:0/3644709043 shutdown_connections 2023-10-16T08:39:13.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.115+0000 7fa53d497640 1 -- 172.21.15.40:0/3644709043 wait complete. 2023-10-16T08:39:13.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.115+0000 7fa53d497640 1 Processor -- start 2023-10-16T08:39:13.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.115+0000 7fa53d497640 1 -- start start 2023-10-16T08:39:13.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.115+0000 7fa53d497640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa520005c30 0x7fa5380b1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:13.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.115+0000 7fa53d497640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa53815a090 0x7fa5380c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:13.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa53d497640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5380c6af0 0x7fa5380c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:13.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa53d497640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5380a8900 con 0x7fa53815a090 2023-10-16T08:39:13.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa53d497640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5380c6120 con 0x7fa520005c30 2023-10-16T08:39:13.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa53d497640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5380c8cf0 con 0x7fa5380c6af0 2023-10-16T08:39:13.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa5367fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa53815a090 0x7fa5380c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:13.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa5367fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa53815a090 0x7fa5380c65b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48172/0 (socket says 172.21.15.40:48172) 2023-10-16T08:39:13.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa5367fc640 1 -- 172.21.15.40:0/1499638593 learned_addr learned my addr 172.21.15.40:0/1499638593 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:13.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa536ffd640 1 --2- 172.21.15.40:0/1499638593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa520005c30 0x7fa5380b1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:13.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa5377fe640 1 --2- 172.21.15.40:0/1499638593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5380c6af0 0x7fa5380c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:13.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa5367fc640 1 -- 172.21.15.40:0/1499638593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5380c6af0 msgr2=0x7fa5380c6f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:13.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa5367fc640 1 --2- 172.21.15.40:0/1499638593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5380c6af0 0x7fa5380c6f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa5367fc640 1 -- 172.21.15.40:0/1499638593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa520005c30 msgr2=0x7fa5380b1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:13.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa5367fc640 1 --2- 172.21.15.40:0/1499638593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa520005c30 0x7fa5380b1ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa5367fc640 1 -- 172.21.15.40:0/1499638593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5380c8730 con 0x7fa53815a090 2023-10-16T08:39:13.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa5367fc640 1 --2- 172.21.15.40:0/1499638593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa53815a090 0x7fa5380c65b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00e4d0 tx=0x7fa52c00e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:13.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa51f7fe640 1 -- 172.21.15.40:0/1499638593 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c0052e0 con 0x7fa53815a090 2023-10-16T08:39:13.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa51f7fe640 1 -- 172.21.15.40:0/1499638593 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa52c005440 con 0x7fa53815a090 2023-10-16T08:39:13.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa53d497640 1 -- 172.21.15.40:0/1499638593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5380c8a10 con 0x7fa53815a090 2023-10-16T08:39:13.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa51f7fe640 1 -- 172.21.15.40:0/1499638593 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa52c003820 con 0x7fa53815a090 2023-10-16T08:39:13.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa53d497640 1 -- 172.21.15.40:0/1499638593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5380c5db0 con 0x7fa53815a090 2023-10-16T08:39:13.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa53d497640 1 -- 172.21.15.40:0/1499638593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5380c6290 con 0x7fa53815a090 2023-10-16T08:39:13.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa51f7fe640 1 -- 172.21.15.40:0/1499638593 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa52c003980 con 0x7fa53815a090 2023-10-16T08:39:13.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa51f7fe640 1 --2- 172.21.15.40:0/1499638593 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa5240664a0 0x7fa524068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:13.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa51f7fe640 1 -- 172.21.15.40:0/1499638593 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa52c083c00 con 0x7fa53815a090 2023-10-16T08:39:13.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa536ffd640 1 --2- 172.21.15.40:0/1499638593 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa5240664a0 0x7fa524068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:13.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.119+0000 7fa536ffd640 1 --2- 172.21.15.40:0/1499638593 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa5240664a0 0x7fa524068960 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa5280ab000 tx=0x7fa5280a7070 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:13.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.123+0000 7fa51f7fe640 1 -- 172.21.15.40:0/1499638593 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fa52c051020 con 0x7fa53815a090 2023-10-16T08:39:13.240 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:39:13.243 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:39:13.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.283+0000 7fa53d497640 1 -- 172.21.15.40:0/1499638593 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5380c5f40 con 0x7fa53815a090 2023-10-16T08:39:13.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.287+0000 7fa51f7fe640 1 -- 172.21.15.40:0/1499638593 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16445 (secure 0 0 0) 0x7fa52c054640 con 0x7fa53815a090 2023-10-16T08:39:13.292 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:39:13.293 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:46.611098+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:39:13.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.287+0000 7fa53d497640 1 -- 172.21.15.40:0/1499638593 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa5240664a0 msgr2=0x7fa524068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:13.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.287+0000 7fa53d497640 1 --2- 172.21.15.40:0/1499638593 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa5240664a0 0x7fa524068960 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fa5280ab000 tx=0x7fa5280a7070 comp rx=0 tx=0).stop 2023-10-16T08:39:13.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.287+0000 7fa53d497640 1 -- 172.21.15.40:0/1499638593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa53815a090 msgr2=0x7fa5380c65b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:13.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.287+0000 7fa53d497640 1 --2- 172.21.15.40:0/1499638593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa53815a090 0x7fa5380c65b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fa52c00e4d0 tx=0x7fa52c00e990 comp rx=0 tx=0).stop 2023-10-16T08:39:13.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.287+0000 7fa53d497640 1 -- 172.21.15.40:0/1499638593 shutdown_connections 2023-10-16T08:39:13.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.287+0000 7fa53d497640 1 --2- 172.21.15.40:0/1499638593 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa5380c6af0 0x7fa5380c6f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.287+0000 7fa53d497640 1 --2- 172.21.15.40:0/1499638593 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa5240664a0 0x7fa524068960 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.287+0000 7fa53d497640 1 --2- 172.21.15.40:0/1499638593 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa53815a090 0x7fa5380c65b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.287+0000 7fa53d497640 1 --2- 172.21.15.40:0/1499638593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa520005c30 0x7fa5380b1ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.287+0000 7fa53d497640 1 -- 172.21.15.40:0/1499638593 >> 172.21.15.40:0/1499638593 conn(0x7fa5380ce960 msgr2=0x7fa5380cb270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:13.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.287+0000 7fa53d497640 1 -- 172.21.15.40:0/1499638593 shutdown_connections 2023-10-16T08:39:13.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.287+0000 7fa53d497640 1 -- 172.21.15.40:0/1499638593 wait complete. 2023-10-16T08:39:13.311 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:38:33.014668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:38:43.055215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:38:43.971666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:39:13.312 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:39:13.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.479+0000 7f3322d09640 1 Processor -- start 2023-10-16T08:39:13.483 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.479+0000 7f3322d09640 1 -- start start 2023-10-16T08:39:13.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.479+0000 7f3322d09640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f331c0adad0 0x7f331c0b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:13.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.479+0000 7f3322d09640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f331c0cae10 0x7f331c15a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:13.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.479+0000 7f3322d09640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f331c0c98b0 con 0x7f331c0adeb0 2023-10-16T08:39:13.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.479+0000 7f3322d09640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f331c0b47c0 con 0x7f331c0cae10 2023-10-16T08:39:13.485 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.479+0000 7f3322d09640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f331c0c8f80 con 0x7f331c0adad0 2023-10-16T08:39:13.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.479+0000 7f33212a6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f331c0cae10 0x7f331c15a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:13.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.479+0000 7f33212a6640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f331c0cae10 0x7f331c15a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48178/0 (socket says 172.21.15.40:48178) 2023-10-16T08:39:13.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.479+0000 7f33212a6640 1 -- 172.21.15.40:0/3646342267 learned_addr learned my addr 172.21.15.40:0/3646342267 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:13.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.479+0000 7f3320aa5640 1 --2- 172.21.15.40:0/3646342267 >> v2:172.21.15.86:3300/0 conn(0x7f331c0adad0 0x7f331c0b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:13.486 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.479+0000 7f33212a6640 1 -- 172.21.15.40:0/3646342267 >> v2:172.21.15.86:3300/0 conn(0x7f331c0adad0 msgr2=0x7f331c0b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:13.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.479+0000 7f33212a6640 1 --2- 172.21.15.40:0/3646342267 >> v2:172.21.15.86:3300/0 conn(0x7f331c0adad0 0x7f331c0b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.479+0000 7f33212a6640 1 -- 172.21.15.40:0/3646342267 >> v1:172.21.15.86:6789/0 conn(0x7f331c0adeb0 legacy=0x7f331c0ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:13.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.479+0000 7f33212a6640 1 -- 172.21.15.40:0/3646342267 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f331c0c9260 con 0x7f331c0cae10 2023-10-16T08:39:13.487 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.479+0000 7f3320aa5640 1 --2- 172.21.15.40:0/3646342267 >> v2:172.21.15.86:3300/0 conn(0x7f331c0adad0 0x7f331c0b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:13.488 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.479+0000 7f33212a6640 1 --2- 172.21.15.40:0/3646342267 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f331c0cae10 0x7f331c15a500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f331c06de30 tx=0x7f331401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fb6bb30ac9e9e89 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:13.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f331b7fe640 1 -- 172.21.15.40:0/3646342267 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33140aa610 con 0x7f331c0cae10 2023-10-16T08:39:13.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f331b7fe640 1 -- 172.21.15.40:0/3646342267 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33140a6040 con 0x7f331c0cae10 2023-10-16T08:39:13.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f331b7fe640 1 -- 172.21.15.40:0/3646342267 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33140aa990 con 0x7f331c0cae10 2023-10-16T08:39:13.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f3322d09640 1 -- 172.21.15.40:0/3646342267 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f331c0cae10 msgr2=0x7f331c15a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:13.489 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f3322d09640 1 --2- 172.21.15.40:0/3646342267 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f331c0cae10 0x7f331c15a500 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f331c06de30 tx=0x7f331401c490 comp rx=0 tx=0).stop 2023-10-16T08:39:13.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f3322d09640 1 -- 172.21.15.40:0/3646342267 shutdown_connections 2023-10-16T08:39:13.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f3322d09640 1 --2- 172.21.15.40:0/3646342267 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f331c0cae10 0x7f331c15a500 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f3322d09640 1 --2- 172.21.15.40:0/3646342267 >> v2:172.21.15.86:3300/0 conn(0x7f331c0adad0 0x7f331c0b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f3322d09640 1 -- 172.21.15.40:0/3646342267 >> 172.21.15.40:0/3646342267 conn(0x7f331c0ce890 msgr2=0x7f331c0ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:13.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f3322d09640 1 -- 172.21.15.40:0/3646342267 shutdown_connections 2023-10-16T08:39:13.490 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f3322d09640 1 -- 172.21.15.40:0/3646342267 wait complete. 2023-10-16T08:39:13.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f3322d09640 1 Processor -- start 2023-10-16T08:39:13.491 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f3322d09640 1 -- start start 2023-10-16T08:39:13.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f3322d09640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f331c0adad0 0x7f331c14eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:13.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f3322d09640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f331c0adeb0 0x7f331c1509b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:13.493 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f3322d09640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f331c0cae10 0x7f331c0c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:13.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f3322d09640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331c14b1f0 con 0x7f331c0adeb0 2023-10-16T08:39:13.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f3322d09640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f331c1501e0 con 0x7f331c0adad0 2023-10-16T08:39:13.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f3322d09640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f331c14f140 con 0x7f331c0cae10 2023-10-16T08:39:13.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f331bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f331c0adeb0 0x7f331c1509b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:13.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f331bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f331c0adeb0 0x7f331c1509b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48184/0 (socket says 172.21.15.40:48184) 2023-10-16T08:39:13.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f331bfff640 1 -- 172.21.15.40:0/4249209208 learned_addr learned my addr 172.21.15.40:0/4249209208 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:13.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f3320aa5640 1 --2- 172.21.15.40:0/4249209208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f331c0adad0 0x7f331c14eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:13.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f33212a6640 1 --2- 172.21.15.40:0/4249209208 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f331c0cae10 0x7f331c0c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:13.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f331bfff640 1 -- 172.21.15.40:0/4249209208 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f331c0cae10 msgr2=0x7f331c0c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:13.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f331bfff640 1 --2- 172.21.15.40:0/4249209208 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f331c0cae10 0x7f331c0c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f331bfff640 1 -- 172.21.15.40:0/4249209208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f331c0adad0 msgr2=0x7f331c14eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:13.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f331bfff640 1 --2- 172.21.15.40:0/4249209208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f331c0adad0 0x7f331c14eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f331bfff640 1 -- 172.21.15.40:0/4249209208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33140094d0 con 0x7f331c0adeb0 2023-10-16T08:39:13.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f33212a6640 1 --2- 172.21.15.40:0/4249209208 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f331c0cae10 0x7f331c0c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:13.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f331bfff640 1 --2- 172.21.15.40:0/4249209208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f331c0adeb0 0x7f331c1509b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f330400c410 tx=0x7f330400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:13.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f33197fa640 1 -- 172.21.15.40:0/4249209208 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304003600 con 0x7f331c0adeb0 2023-10-16T08:39:13.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f33197fa640 1 -- 172.21.15.40:0/4249209208 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3304003780 con 0x7f331c0adeb0 2023-10-16T08:39:13.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f33197fa640 1 -- 172.21.15.40:0/4249209208 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3304003b30 con 0x7f331c0adeb0 2023-10-16T08:39:13.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.483+0000 7f3322d09640 1 -- 172.21.15.40:0/4249209208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f331c15b250 con 0x7f331c0adeb0 2023-10-16T08:39:13.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.487+0000 7f3322d09640 1 -- 172.21.15.40:0/4249209208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f331c14f700 con 0x7f331c0adeb0 2023-10-16T08:39:13.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.487+0000 7f3322d09640 1 -- 172.21.15.40:0/4249209208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f331c067a20 con 0x7f331c0adeb0 2023-10-16T08:39:13.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.487+0000 7f33197fa640 1 -- 172.21.15.40:0/4249209208 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3304003c90 con 0x7f331c0adeb0 2023-10-16T08:39:13.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.487+0000 7f33197fa640 1 --2- 172.21.15.40:0/4249209208 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f32fc0664a0 0x7f32fc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:13.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.487+0000 7f33197fa640 1 -- 172.21.15.40:0/4249209208 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3304082ff0 con 0x7f331c0adeb0 2023-10-16T08:39:13.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.487+0000 7f3320aa5640 1 --2- 172.21.15.40:0/4249209208 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f32fc0664a0 0x7f32fc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:13.498 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.487+0000 7f3320aa5640 1 --2- 172.21.15.40:0/4249209208 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f32fc0664a0 0x7f32fc068960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f330c008f80 tx=0x7f330c0084d0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:13.499 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.491+0000 7f33197fa640 1 -- 172.21.15.40:0/4249209208 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f3304050520 con 0x7f331c0adeb0 2023-10-16T08:39:13.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.651+0000 7f3322d09640 1 -- 172.21.15.40:0/4249209208 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f331c04e9c0 con 0x7f331c0adeb0 2023-10-16T08:39:13.658 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.651+0000 7f33197fa640 1 -- 172.21.15.40:0/4249209208 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3304053b40 con 0x7f331c0adeb0 2023-10-16T08:39:13.659 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-16T08:39:13.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.655+0000 7f3322d09640 1 -- 172.21.15.40:0/4249209208 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f32fc0664a0 msgr2=0x7f32fc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:13.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.655+0000 7f3322d09640 1 --2- 172.21.15.40:0/4249209208 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f32fc0664a0 0x7f32fc068960 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f330c008f80 tx=0x7f330c0084d0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.655+0000 7f3322d09640 1 -- 172.21.15.40:0/4249209208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f331c0adeb0 msgr2=0x7f331c1509b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:13.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.655+0000 7f3322d09640 1 --2- 172.21.15.40:0/4249209208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f331c0adeb0 0x7f331c1509b0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f330400c410 tx=0x7f330400c8d0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.655+0000 7f3322d09640 1 -- 172.21.15.40:0/4249209208 shutdown_connections 2023-10-16T08:39:13.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.655+0000 7f3322d09640 1 --2- 172.21.15.40:0/4249209208 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f331c0cae10 0x7f331c0c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.655+0000 7f3322d09640 1 --2- 172.21.15.40:0/4249209208 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f32fc0664a0 0x7f32fc068960 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.655+0000 7f3322d09640 1 --2- 172.21.15.40:0/4249209208 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f331c0adeb0 0x7f331c1509b0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.655+0000 7f3322d09640 1 --2- 172.21.15.40:0/4249209208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f331c0adad0 0x7f331c14eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.655+0000 7f3322d09640 1 -- 172.21.15.40:0/4249209208 >> 172.21.15.40:0/4249209208 conn(0x7f331c0ce890 msgr2=0x7f331c0c4570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:13.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.655+0000 7f3322d09640 1 -- 172.21.15.40:0/4249209208 shutdown_connections 2023-10-16T08:39:13.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.655+0000 7f3322d09640 1 -- 172.21.15.40:0/4249209208 wait complete. 2023-10-16T08:39:13.679 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:39:13.803 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.799+0000 7f4df0855640 1 Processor -- start 2023-10-16T08:39:13.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.799+0000 7f4df0855640 1 -- start start 2023-10-16T08:39:13.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.799+0000 7f4df0855640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f4dec0ce540 0x7f4dec0cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:13.805 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.799+0000 7f4df0855640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4dec0d02c0 0x7f4dec164750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:13.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.799+0000 7f4df0855640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dec0cd2e0 con 0x7f4dec0afb90 2023-10-16T08:39:13.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.799+0000 7f4df0855640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dec0cf930 con 0x7f4dec0d02c0 2023-10-16T08:39:13.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.799+0000 7f4df0855640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dec0cc6d0 con 0x7f4dec0ce540 2023-10-16T08:39:13.806 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.799+0000 7f4dead9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4dec0d02c0 0x7f4dec164750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:13.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.799+0000 7f4dead9d640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f4dec0d02c0 0x7f4dec164750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48186/0 (socket says 172.21.15.40:48186) 2023-10-16T08:39:13.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.799+0000 7f4dead9d640 1 -- 172.21.15.40:0/3002411569 learned_addr learned my addr 172.21.15.40:0/3002411569 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:13.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.799+0000 7f4dea59c640 1 --2- 172.21.15.40:0/3002411569 >> v2:172.21.15.86:3300/0 conn(0x7f4dec0ce540 0x7f4dec0cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:13.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.799+0000 7f4dead9d640 1 -- 172.21.15.40:0/3002411569 >> v2:172.21.15.86:3300/0 conn(0x7f4dec0ce540 msgr2=0x7f4dec0cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:13.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.799+0000 7f4dead9d640 1 --2- 172.21.15.40:0/3002411569 >> v2:172.21.15.86:3300/0 conn(0x7f4dec0ce540 0x7f4dec0cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.799+0000 7f4dead9d640 1 -- 172.21.15.40:0/3002411569 >> v1:172.21.15.86:6789/0 conn(0x7f4dec0afb90 legacy=0x7f4dec0cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:13.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.799+0000 7f4dead9d640 1 -- 172.21.15.40:0/3002411569 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dec0cc9b0 con 0x7f4dec0d02c0 2023-10-16T08:39:13.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4dead9d640 1 --2- 172.21.15.40:0/3002411569 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dec0d02c0 0x7f4dec164750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4dec0d1a00 tx=0x7f4de401c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4fda9789f0e6d2c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:13.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4de959a640 1 -- 172.21.15.40:0/3002411569 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de40a5870 con 0x7f4dec0d02c0 2023-10-16T08:39:13.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4de959a640 1 -- 172.21.15.40:0/3002411569 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de40a59d0 con 0x7f4dec0d02c0 2023-10-16T08:39:13.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4de959a640 1 -- 172.21.15.40:0/3002411569 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de40a5d60 con 0x7f4dec0d02c0 2023-10-16T08:39:13.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4df0855640 1 -- 172.21.15.40:0/3002411569 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dec0d02c0 msgr2=0x7f4dec164750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:13.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4df0855640 1 --2- 172.21.15.40:0/3002411569 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dec0d02c0 0x7f4dec164750 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4dec0d1a00 tx=0x7f4de401c990 comp rx=0 tx=0).stop 2023-10-16T08:39:13.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4df0855640 1 -- 172.21.15.40:0/3002411569 shutdown_connections 2023-10-16T08:39:13.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4df0855640 1 --2- 172.21.15.40:0/3002411569 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dec0d02c0 0x7f4dec164750 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4df0855640 1 --2- 172.21.15.40:0/3002411569 >> v2:172.21.15.86:3300/0 conn(0x7f4dec0ce540 0x7f4dec0cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4df0855640 1 -- 172.21.15.40:0/3002411569 >> 172.21.15.40:0/3002411569 conn(0x7f4dec065e80 msgr2=0x7f4dec066290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:13.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4df0855640 1 -- 172.21.15.40:0/3002411569 shutdown_connections 2023-10-16T08:39:13.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4df0855640 1 -- 172.21.15.40:0/3002411569 wait complete. 2023-10-16T08:39:13.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4df0855640 1 Processor -- start 2023-10-16T08:39:13.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4df0855640 1 -- start start 2023-10-16T08:39:13.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4df0855640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dec0afb90 0x7f4dec15bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:13.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4df0855640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dec0ce540 0x7f4dec06ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:13.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4df0855640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4dec0d02c0 0x7f4dec150f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4df0855640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec06bf60 con 0x7f4dec0ce540 2023-10-16T08:39:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4df0855640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec15cf70 con 0x7f4dec0afb90 2023-10-16T08:39:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4df0855640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dec15c360 con 0x7f4dec0d02c0 2023-10-16T08:39:13.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4de9d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dec0ce540 0x7f4dec06ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4de9d9b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dec0ce540 0x7f4dec06ba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48198/0 (socket says 172.21.15.40:48198) 2023-10-16T08:39:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4de9d9b640 1 -- 172.21.15.40:0/1336294046 learned_addr learned my addr 172.21.15.40:0/1336294046 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4dea59c640 1 --2- 172.21.15.40:0/1336294046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dec0afb90 0x7f4dec15bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4dead9d640 1 --2- 172.21.15.40:0/1336294046 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4dec0d02c0 0x7f4dec150f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4de9d9b640 1 -- 172.21.15.40:0/1336294046 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4dec0d02c0 msgr2=0x7f4dec150f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:13.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4de9d9b640 1 --2- 172.21.15.40:0/1336294046 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4dec0d02c0 0x7f4dec150f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4de9d9b640 1 -- 172.21.15.40:0/1336294046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dec0afb90 msgr2=0x7f4dec15bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:13.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4de9d9b640 1 --2- 172.21.15.40:0/1336294046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dec0afb90 0x7f4dec15bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4de9d9b640 1 -- 172.21.15.40:0/1336294046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4de4009b30 con 0x7f4dec0ce540 2023-10-16T08:39:13.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.803+0000 7f4dead9d640 1 --2- 172.21.15.40:0/1336294046 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4dec0d02c0 0x7f4dec150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:13.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.807+0000 7f4de9d9b640 1 --2- 172.21.15.40:0/1336294046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dec0ce540 0x7f4dec06ba20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4de000b440 tx=0x7f4de000b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:13.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.807+0000 7f4ddaffd640 1 -- 172.21.15.40:0/1336294046 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0011890 con 0x7f4dec0ce540 2023-10-16T08:39:13.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.807+0000 7f4ddaffd640 1 -- 172.21.15.40:0/1336294046 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4de00119f0 con 0x7f4dec0ce540 2023-10-16T08:39:13.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.807+0000 7f4df0855640 1 -- 172.21.15.40:0/1336294046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dec0cde30 con 0x7f4dec0ce540 2023-10-16T08:39:13.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.807+0000 7f4ddaffd640 1 -- 172.21.15.40:0/1336294046 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4de0011da0 con 0x7f4dec0ce540 2023-10-16T08:39:13.818 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.807+0000 7f4df0855640 1 -- 172.21.15.40:0/1336294046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dec15c920 con 0x7f4dec0ce540 2023-10-16T08:39:13.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.807+0000 7f4df0855640 1 -- 172.21.15.40:0/1336294046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dec15d100 con 0x7f4dec0ce540 2023-10-16T08:39:13.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.807+0000 7f4ddaffd640 1 -- 172.21.15.40:0/1336294046 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f4de001a400 con 0x7f4dec0ce540 2023-10-16T08:39:13.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.807+0000 7f4ddaffd640 1 --2- 172.21.15.40:0/1336294046 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f4dcc0664d0 0x7f4dcc068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:13.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.807+0000 7f4ddaffd640 1 -- 172.21.15.40:0/1336294046 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4de00841f0 con 0x7f4dec0ce540 2023-10-16T08:39:13.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.807+0000 7f4dea59c640 1 --2- 172.21.15.40:0/1336294046 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f4dcc0664d0 0x7f4dcc068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:13.819 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.807+0000 7f4dea59c640 1 --2- 172.21.15.40:0/1336294046 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f4dcc0664d0 0x7f4dcc068990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc00b9d0 tx=0x7f4ddc008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:13.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.811+0000 7f4ddaffd640 1 -- 172.21.15.40:0/1336294046 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f4de0051670 con 0x7f4dec0ce540 2023-10-16T08:39:13.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.963+0000 7f4df0855640 1 -- 172.21.15.40:0/1336294046 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f4dec04e9c0 con 0x7f4dec0ce540 2023-10-16T08:39:13.967 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.963+0000 7f4ddaffd640 1 -- 172.21.15.40:0/1336294046 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4de0054c90 con 0x7f4dec0ce540 2023-10-16T08:39:13.968 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-16T08:39:13.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.963+0000 7f4df0855640 1 -- 172.21.15.40:0/1336294046 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f4dcc0664d0 msgr2=0x7f4dcc068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:13.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.963+0000 7f4df0855640 1 --2- 172.21.15.40:0/1336294046 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f4dcc0664d0 0x7f4dcc068990 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f4ddc00b9d0 tx=0x7f4ddc008040 comp rx=0 tx=0).stop 2023-10-16T08:39:13.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.963+0000 7f4df0855640 1 -- 172.21.15.40:0/1336294046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dec0ce540 msgr2=0x7f4dec06ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:13.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.963+0000 7f4df0855640 1 --2- 172.21.15.40:0/1336294046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dec0ce540 0x7f4dec06ba20 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f4de000b440 tx=0x7f4de000b900 comp rx=0 tx=0).stop 2023-10-16T08:39:13.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.963+0000 7f4df0855640 1 -- 172.21.15.40:0/1336294046 shutdown_connections 2023-10-16T08:39:13.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.963+0000 7f4df0855640 1 --2- 172.21.15.40:0/1336294046 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f4dec0d02c0 0x7f4dec150f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.963+0000 7f4df0855640 1 --2- 172.21.15.40:0/1336294046 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f4dcc0664d0 0x7f4dcc068990 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.963+0000 7f4df0855640 1 --2- 172.21.15.40:0/1336294046 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f4dec0ce540 0x7f4dec06ba20 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.963+0000 7f4df0855640 1 --2- 172.21.15.40:0/1336294046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4dec0afb90 0x7f4dec15bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:13.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.963+0000 7f4df0855640 1 -- 172.21.15.40:0/1336294046 >> 172.21.15.40:0/1336294046 conn(0x7f4dec065e80 msgr2=0x7f4dec06d700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:13.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.963+0000 7f4df0855640 1 -- 172.21.15.40:0/1336294046 shutdown_connections 2023-10-16T08:39:13.972 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:13.963+0000 7f4df0855640 1 -- 172.21.15.40:0/1336294046 wait complete. 2023-10-16T08:39:13.985 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:39:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.103+0000 7fced8e17640 1 Processor -- start 2023-10-16T08:39:14.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.103+0000 7fced8e17640 1 -- start start 2023-10-16T08:39:14.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.107+0000 7fced8e17640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fced4159dd0 0x7fced40c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:14.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.107+0000 7fced8e17640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fced40c9f00 0x7fced40cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:14.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.107+0000 7fced8e17640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fced415b490 con 0x7fced40adec0 2023-10-16T08:39:14.114 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.107+0000 7fced8e17640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fced406cd70 con 0x7fced4159dd0 2023-10-16T08:39:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.107+0000 7fced8e17640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced406d230 con 0x7fced40c9f00 2023-10-16T08:39:14.115 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.107+0000 7fced2d9d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fced40c9f00 0x7fced40cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.107+0000 7fced2d9d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fced40c9f00 0x7fced40cc460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3301/0 says I am v2:172.21.15.40:42052/0 (socket says 172.21.15.40:42052) 2023-10-16T08:39:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.107+0000 7fced259c640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fced4159dd0 0x7fced40c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:14.116 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.107+0000 7fced2d9d640 1 -- 172.21.15.40:0/2937424839 learned_addr learned my addr 172.21.15.40:0/2937424839 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:14.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.107+0000 7fced2d9d640 1 -- 172.21.15.40:0/2937424839 >> v2:172.21.15.86:3300/0 conn(0x7fced4159dd0 msgr2=0x7fced40c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:14.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.107+0000 7fced2d9d640 1 --2- 172.21.15.40:0/2937424839 >> v2:172.21.15.86:3300/0 conn(0x7fced4159dd0 0x7fced40c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.107+0000 7fced2d9d640 1 -- 172.21.15.40:0/2937424839 >> v1:172.21.15.86:6789/0 conn(0x7fced40adec0 legacy=0x7fced40c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:14.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.107+0000 7fced2d9d640 1 -- 172.21.15.40:0/2937424839 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fced406cf50 con 0x7fced40c9f00 2023-10-16T08:39:14.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced2d9d640 1 --2- 172.21.15.40:0/2937424839 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fced40c9f00 0x7fced40cc460 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009e50 tx=0x7fcecc01ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfa0625af6b3d10e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:14.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced159a640 1 -- 172.21.15.40:0/2937424839 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc0a57f0 con 0x7fced40c9f00 2023-10-16T08:39:14.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced159a640 1 -- 172.21.15.40:0/2937424839 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc0a5950 con 0x7fced40c9f00 2023-10-16T08:39:14.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced159a640 1 -- 172.21.15.40:0/2937424839 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc0a5ce0 con 0x7fced40c9f00 2023-10-16T08:39:14.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced8e17640 1 -- 172.21.15.40:0/2937424839 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fced40c9f00 msgr2=0x7fced40cc460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:14.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced8e17640 1 --2- 172.21.15.40:0/2937424839 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fced40c9f00 0x7fced40cc460 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcecc009e50 tx=0x7fcecc01ca10 comp rx=0 tx=0).stop 2023-10-16T08:39:14.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced8e17640 1 -- 172.21.15.40:0/2937424839 shutdown_connections 2023-10-16T08:39:14.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced8e17640 1 --2- 172.21.15.40:0/2937424839 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fced40c9f00 0x7fced40cc460 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced8e17640 1 --2- 172.21.15.40:0/2937424839 >> v2:172.21.15.86:3300/0 conn(0x7fced4159dd0 0x7fced40c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced8e17640 1 -- 172.21.15.40:0/2937424839 >> 172.21.15.40:0/2937424839 conn(0x7fced40ce890 msgr2=0x7fced40ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:14.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced8e17640 1 -- 172.21.15.40:0/2937424839 shutdown_connections 2023-10-16T08:39:14.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced8e17640 1 -- 172.21.15.40:0/2937424839 wait complete. 2023-10-16T08:39:14.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced8e17640 1 Processor -- start 2023-10-16T08:39:14.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced8e17640 1 -- start start 2023-10-16T08:39:14.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced8e17640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fced40adec0 0x7fced414c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:14.130 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced8e17640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fced40c9f00 0x7fced406fa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:14.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced8e17640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fced4159dd0 0x7fced40c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:14.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced8e17640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced4070ef0 con 0x7fced4159dd0 2023-10-16T08:39:14.131 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced8e17640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fced41536a0 con 0x7fced40c9f00 2023-10-16T08:39:14.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced8e17640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fced41527b0 con 0x7fced40adec0 2023-10-16T08:39:14.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fced4159dd0 0x7fced40c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:14.132 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced2d9d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fced4159dd0 0x7fced40c7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48202/0 (socket says 172.21.15.40:48202) 2023-10-16T08:39:14.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced2d9d640 1 -- 172.21.15.40:0/3354817791 learned_addr learned my addr 172.21.15.40:0/3354817791 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:14.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced1d9b640 1 --2- 172.21.15.40:0/3354817791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fced40c9f00 0x7fced406fa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:14.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced259c640 1 --2- 172.21.15.40:0/3354817791 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fced40adec0 0x7fced414c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:14.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced2d9d640 1 -- 172.21.15.40:0/3354817791 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fced40adec0 msgr2=0x7fced414c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:14.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced2d9d640 1 --2- 172.21.15.40:0/3354817791 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fced40adec0 0x7fced414c0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced2d9d640 1 -- 172.21.15.40:0/3354817791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fced40c9f00 msgr2=0x7fced406fa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:14.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced2d9d640 1 --2- 172.21.15.40:0/3354817791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fced40c9f00 0x7fced406fa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced2d9d640 1 -- 172.21.15.40:0/3354817791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcecc009b30 con 0x7fced4159dd0 2023-10-16T08:39:14.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced2d9d640 1 --2- 172.21.15.40:0/3354817791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fced4159dd0 0x7fced40c7fa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcecc01a640 tx=0x7fcecc0197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:14.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fcec2ffd640 1 -- 172.21.15.40:0/3354817791 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc0b8670 con 0x7fced4159dd0 2023-10-16T08:39:14.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fcec2ffd640 1 -- 172.21.15.40:0/3354817791 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcecc0b87d0 con 0x7fced4159dd0 2023-10-16T08:39:14.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fced8e17640 1 -- 172.21.15.40:0/3354817791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fced4152a90 con 0x7fced4159dd0 2023-10-16T08:39:14.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.111+0000 7fcec2ffd640 1 -- 172.21.15.40:0/3354817791 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcecc0b8b40 con 0x7fced4159dd0 2023-10-16T08:39:14.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.115+0000 7fced8e17640 1 -- 172.21.15.40:0/3354817791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fced4153050 con 0x7fced4159dd0 2023-10-16T08:39:14.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.115+0000 7fced8e17640 1 -- 172.21.15.40:0/3354817791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fced4067a20 con 0x7fced4159dd0 2023-10-16T08:39:14.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.115+0000 7fcec2ffd640 1 -- 172.21.15.40:0/3354817791 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcecc0b8ca0 con 0x7fced4159dd0 2023-10-16T08:39:14.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.115+0000 7fcec2ffd640 1 --2- 172.21.15.40:0/3354817791 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fceb4066470 0x7fceb4068930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:14.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.115+0000 7fcec2ffd640 1 -- 172.21.15.40:0/3354817791 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcecc11f1d0 con 0x7fced4159dd0 2023-10-16T08:39:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.115+0000 7fced259c640 1 --2- 172.21.15.40:0/3354817791 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fceb4066470 0x7fceb4068930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.115+0000 7fced259c640 1 --2- 172.21.15.40:0/3354817791 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fceb4066470 0x7fceb4068930 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009a50 tx=0x7fcec4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:14.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.119+0000 7fcec2ffd640 1 -- 172.21.15.40:0/3354817791 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fcecc0ec6e0 con 0x7fced4159dd0 2023-10-16T08:39:14.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.275+0000 7fced8e17640 1 -- 172.21.15.40:0/3354817791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fced40ce660 con 0x7fced4159dd0 2023-10-16T08:39:14.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.275+0000 7fcec2ffd640 1 -- 172.21.15.40:0/3354817791 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcecc0efd00 con 0x7fced4159dd0 2023-10-16T08:39:14.281 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-16T08:39:14.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.275+0000 7fced8e17640 1 -- 172.21.15.40:0/3354817791 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fceb4066470 msgr2=0x7fceb4068930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:14.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.275+0000 7fced8e17640 1 --2- 172.21.15.40:0/3354817791 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fceb4066470 0x7fceb4068930 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fcec4009a50 tx=0x7fcec4008040 comp rx=0 tx=0).stop 2023-10-16T08:39:14.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.275+0000 7fced8e17640 1 -- 172.21.15.40:0/3354817791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fced4159dd0 msgr2=0x7fced40c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:14.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.275+0000 7fced8e17640 1 --2- 172.21.15.40:0/3354817791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fced4159dd0 0x7fced40c7fa0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcecc01a640 tx=0x7fcecc0197b0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.275+0000 7fced8e17640 1 -- 172.21.15.40:0/3354817791 shutdown_connections 2023-10-16T08:39:14.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.275+0000 7fced8e17640 1 --2- 172.21.15.40:0/3354817791 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fceb4066470 0x7fceb4068930 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.275+0000 7fced8e17640 1 --2- 172.21.15.40:0/3354817791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fced4159dd0 0x7fced40c7fa0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.275+0000 7fced8e17640 1 --2- 172.21.15.40:0/3354817791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fced40c9f00 0x7fced406fa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.275+0000 7fced8e17640 1 --2- 172.21.15.40:0/3354817791 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fced40adec0 0x7fced414c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.275+0000 7fced8e17640 1 -- 172.21.15.40:0/3354817791 >> 172.21.15.40:0/3354817791 conn(0x7fced40ce890 msgr2=0x7fced4158e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:14.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.275+0000 7fced8e17640 1 -- 172.21.15.40:0/3354817791 shutdown_connections 2023-10-16T08:39:14.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.275+0000 7fced8e17640 1 -- 172.21.15.40:0/3354817791 wait complete. 2023-10-16T08:39:14.299 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:39:14.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.403+0000 7f67e2d68640 1 Processor -- start 2023-10-16T08:39:14.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67e2d68640 1 -- start start 2023-10-16T08:39:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67e2d68640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67dc160450 0x7f67dc0ce920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:14.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67e2d68640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f67dc0ccc40 0x7f67dc0cba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67e2d68640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f67dc0cbf80 0x7f67dc0ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67e2d68640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67dc0cde70 con 0x7f67dc0cbf80 2023-10-16T08:39:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67e2d68640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67dc0cd0f0 con 0x7f67dc0ccc40 2023-10-16T08:39:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67e2d68640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc0cc410 con 0x7f67dc160450 2023-10-16T08:39:14.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67e1305640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f67dc0cbf80 0x7f67dc0ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67e1305640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f67dc0cbf80 0x7f67dc0ac8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48212/0 (socket says 172.21.15.40:48212) 2023-10-16T08:39:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67e1305640 1 -- 172.21.15.40:0/2986417438 learned_addr learned my addr 172.21.15.40:0/2986417438 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67e0b04640 1 --2- 172.21.15.40:0/2986417438 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67dc160450 0x7f67dc0ce920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67dbfff640 1 --2- 172.21.15.40:0/2986417438 >> v2:172.21.15.86:3300/0 conn(0x7f67dc0ccc40 0x7f67dc0cba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:14.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67e1305640 1 -- 172.21.15.40:0/2986417438 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67dc160450 msgr2=0x7f67dc0ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67e1305640 1 --2- 172.21.15.40:0/2986417438 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67dc160450 0x7f67dc0ce920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67e1305640 1 -- 172.21.15.40:0/2986417438 >> v2:172.21.15.86:3300/0 conn(0x7f67dc0ccc40 msgr2=0x7f67dc0cba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67e1305640 1 --2- 172.21.15.40:0/2986417438 >> v2:172.21.15.86:3300/0 conn(0x7f67dc0ccc40 0x7f67dc0cba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67e1305640 1 -- 172.21.15.40:0/2986417438 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67dc160830 con 0x7f67dc0cbf80 2023-10-16T08:39:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67e1305640 1 --2- 172.21.15.40:0/2986417438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67dc0cbf80 0x7f67dc0ac8e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f67dc06d290 tx=0x7f67cc01c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d60a96e54a4a7ae2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67db7fe640 1 -- 172.21.15.40:0/2986417438 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc0a64e0 con 0x7f67dc0cbf80 2023-10-16T08:39:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67db7fe640 1 -- 172.21.15.40:0/2986417438 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67cc0a6640 con 0x7f67dc0cbf80 2023-10-16T08:39:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.407+0000 7f67db7fe640 1 -- 172.21.15.40:0/2986417438 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67cc0a69d0 con 0x7f67dc0cbf80 2023-10-16T08:39:14.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e2d68640 1 -- 172.21.15.40:0/2986417438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67dc0cbf80 msgr2=0x7f67dc0ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:14.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e2d68640 1 --2- 172.21.15.40:0/2986417438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67dc0cbf80 0x7f67dc0ac8e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f67dc06d290 tx=0x7f67cc01c650 comp rx=0 tx=0).stop 2023-10-16T08:39:14.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e2d68640 1 -- 172.21.15.40:0/2986417438 shutdown_connections 2023-10-16T08:39:14.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e2d68640 1 --2- 172.21.15.40:0/2986417438 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67dc0cbf80 0x7f67dc0ac8e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e2d68640 1 --2- 172.21.15.40:0/2986417438 >> v2:172.21.15.86:3300/0 conn(0x7f67dc0ccc40 0x7f67dc0cba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e2d68640 1 --2- 172.21.15.40:0/2986417438 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67dc160450 0x7f67dc0ce920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e2d68640 1 -- 172.21.15.40:0/2986417438 >> 172.21.15.40:0/2986417438 conn(0x7f67dc0b0be0 msgr2=0x7f67dc0b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:14.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e2d68640 1 -- 172.21.15.40:0/2986417438 shutdown_connections 2023-10-16T08:39:14.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e2d68640 1 -- 172.21.15.40:0/2986417438 wait complete. 2023-10-16T08:39:14.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e2d68640 1 Processor -- start 2023-10-16T08:39:14.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e2d68640 1 -- start start 2023-10-16T08:39:14.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e2d68640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67dc0cbf80 0x7f67dc0c7890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:14.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e2d68640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67dc0ccc40 0x7f67dc15b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:14.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e2d68640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67dc160450 0x7f67dc070d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:14.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e2d68640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc0c3b70 con 0x7f67dc0ccc40 2023-10-16T08:39:14.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e2d68640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc0c3e50 con 0x7f67dc160450 2023-10-16T08:39:14.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e2d68640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67dc0c4130 con 0x7f67dc0cbf80 2023-10-16T08:39:14.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67dbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67dc0ccc40 0x7f67dc15b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:14.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67dbfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67dc0ccc40 0x7f67dc15b850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:48218/0 (socket says 172.21.15.40:48218) 2023-10-16T08:39:14.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67dbfff640 1 -- 172.21.15.40:0/1869129314 learned_addr learned my addr 172.21.15.40:0/1869129314 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:14.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e0b04640 1 --2- 172.21.15.40:0/1869129314 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67dc0cbf80 0x7f67dc0c7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e1305640 1 --2- 172.21.15.40:0/1869129314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67dc160450 0x7f67dc070d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67dbfff640 1 -- 172.21.15.40:0/1869129314 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67dc0cbf80 msgr2=0x7f67dc0c7890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67dbfff640 1 --2- 172.21.15.40:0/1869129314 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67dc0cbf80 0x7f67dc0c7890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67dbfff640 1 -- 172.21.15.40:0/1869129314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67dc160450 msgr2=0x7f67dc070d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67dbfff640 1 --2- 172.21.15.40:0/1869129314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67dc160450 0x7f67dc070d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67dbfff640 1 -- 172.21.15.40:0/1869129314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67cc009b30 con 0x7f67dc0ccc40 2023-10-16T08:39:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67dbfff640 1 --2- 172.21.15.40:0/1869129314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67dc0ccc40 0x7f67dc15b850 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f67c400c5c0 tx=0x7f67c400ca80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67d97fa640 1 -- 172.21.15.40:0/1869129314 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c4011800 con 0x7f67dc0ccc40 2023-10-16T08:39:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67d97fa640 1 -- 172.21.15.40:0/1869129314 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c4011960 con 0x7f67dc0ccc40 2023-10-16T08:39:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e2d68640 1 -- 172.21.15.40:0/1869129314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67dc0c4410 con 0x7f67dc0ccc40 2023-10-16T08:39:14.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67d97fa640 1 -- 172.21.15.40:0/1869129314 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c4011cf0 con 0x7f67dc0ccc40 2023-10-16T08:39:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.411+0000 7f67e2d68640 1 -- 172.21.15.40:0/1869129314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67dc0c46f0 con 0x7f67dc0ccc40 2023-10-16T08:39:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.415+0000 7f67e2d68640 1 -- 172.21.15.40:0/1869129314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67dc073570 con 0x7f67dc0ccc40 2023-10-16T08:39:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.415+0000 7f67d97fa640 1 -- 172.21.15.40:0/1869129314 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f67c4018360 con 0x7f67dc0ccc40 2023-10-16T08:39:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.415+0000 7f67d97fa640 1 --2- 172.21.15.40:0/1869129314 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f67bc0664a0 0x7f67bc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:14.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.415+0000 7f67d97fa640 1 -- 172.21.15.40:0/1869129314 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f67c4083e90 con 0x7f67dc0ccc40 2023-10-16T08:39:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.415+0000 7f67e0b04640 1 --2- 172.21.15.40:0/1869129314 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f67bc0664a0 0x7f67bc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.415+0000 7f67e0b04640 1 --2- 172.21.15.40:0/1869129314 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f67bc0664a0 0x7f67bc068960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f67d400a8d0 tx=0x7f67d4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:14.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.419+0000 7f67d97fa640 1 -- 172.21.15.40:0/1869129314 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f67c4051300 con 0x7f67dc0ccc40 2023-10-16T08:39:14.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.571+0000 7f67e2d68640 1 -- 172.21.15.40:0/1869129314 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67dc04e9c0 con 0x7f67dc0ccc40 2023-10-16T08:39:14.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.571+0000 7f67d97fa640 1 -- 172.21.15.40:0/1869129314 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f67c4054920 con 0x7f67dc0ccc40 2023-10-16T08:39:14.579 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:39:14.579 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:46.611098+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:39:14.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.575+0000 7f67e2d68640 1 -- 172.21.15.40:0/1869129314 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f67bc0664a0 msgr2=0x7f67bc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:14.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.575+0000 7f67e2d68640 1 --2- 172.21.15.40:0/1869129314 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f67bc0664a0 0x7f67bc068960 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f67d400a8d0 tx=0x7f67d4008040 comp rx=0 tx=0).stop 2023-10-16T08:39:14.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.575+0000 7f67e2d68640 1 -- 172.21.15.40:0/1869129314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67dc0ccc40 msgr2=0x7f67dc15b850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:14.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.575+0000 7f67e2d68640 1 --2- 172.21.15.40:0/1869129314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67dc0ccc40 0x7f67dc15b850 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f67c400c5c0 tx=0x7f67c400ca80 comp rx=0 tx=0).stop 2023-10-16T08:39:14.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.575+0000 7f67e2d68640 1 -- 172.21.15.40:0/1869129314 shutdown_connections 2023-10-16T08:39:14.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.575+0000 7f67e2d68640 1 --2- 172.21.15.40:0/1869129314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67dc160450 0x7f67dc070d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.575+0000 7f67e2d68640 1 --2- 172.21.15.40:0/1869129314 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f67bc0664a0 0x7f67bc068960 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.575+0000 7f67e2d68640 1 --2- 172.21.15.40:0/1869129314 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f67dc0ccc40 0x7f67dc15b850 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.575+0000 7f67e2d68640 1 --2- 172.21.15.40:0/1869129314 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67dc0cbf80 0x7f67dc0c7890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.575+0000 7f67e2d68640 1 -- 172.21.15.40:0/1869129314 >> 172.21.15.40:0/1869129314 conn(0x7f67dc0b0be0 msgr2=0x7f67dc0caa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:14.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.575+0000 7f67e2d68640 1 -- 172.21.15.40:0/1869129314 shutdown_connections 2023-10-16T08:39:14.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.575+0000 7f67e2d68640 1 -- 172.21.15.40:0/1869129314 wait complete. 2023-10-16T08:39:14.596 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:38:33.014668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:38:43.055215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:38:43.971666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:39:14.597 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:39:14.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.715+0000 7fe80f957640 1 Processor -- start 2023-10-16T08:39:14.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.715+0000 7fe80f957640 1 -- start start 2023-10-16T08:39:14.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.715+0000 7fe80f957640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe80815ad00 0x7fe8080cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:14.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.715+0000 7fe80f957640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe808066190 0x7fe8080707e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:14.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.715+0000 7fe80f957640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8080b1b30 con 0x7fe8080cb800 2023-10-16T08:39:14.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.715+0000 7fe80f957640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8080cbc80 con 0x7fe808066190 2023-10-16T08:39:14.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.715+0000 7fe80f957640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8080b12d0 con 0x7fe80815ad00 2023-10-16T08:39:14.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.715+0000 7fe80cef2640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe8080cb800 0x7fe808065a80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60846/0 (socket says 172.21.15.40:60846) 2023-10-16T08:39:14.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.715+0000 7fe80cef2640 1 -- 172.21.15.40:0/2738254182 learned_addr learned my addr 172.21.15.40:0/2738254182 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:14.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.715+0000 7fe80def4640 1 --2- 172.21.15.40:0/2738254182 >> v2:172.21.15.40:3300/0 conn(0x7fe808066190 0x7fe8080707e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:14.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.715+0000 7fe80d6f3640 1 --2- 172.21.15.40:0/2738254182 >> v2:172.21.15.86:3300/0 conn(0x7fe80815ad00 0x7fe8080cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:14.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.715+0000 7fe7fffff640 1 -- 172.21.15.40:0/2738254182 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3358000841 0 0) 0x7fe8080b1b30 con 0x7fe8080cb800 2023-10-16T08:39:14.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.715+0000 7fe7fffff640 1 -- 172.21.15.40:0/2738254182 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7f4003530 con 0x7fe8080cb800 2023-10-16T08:39:14.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.715+0000 7fe80def4640 1 -- 172.21.15.40:0/2738254182 >> v2:172.21.15.86:3300/0 conn(0x7fe80815ad00 msgr2=0x7fe8080cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:14.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.715+0000 7fe80def4640 1 --2- 172.21.15.40:0/2738254182 >> v2:172.21.15.86:3300/0 conn(0x7fe80815ad00 0x7fe8080cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.715+0000 7fe80def4640 1 -- 172.21.15.40:0/2738254182 >> v1:172.21.15.40:6789/0 conn(0x7fe8080cb800 legacy=0x7fe808065a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:14.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.715+0000 7fe80def4640 1 -- 172.21.15.40:0/2738254182 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8080cbe80 con 0x7fe808066190 2023-10-16T08:39:14.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80def4640 1 --2- 172.21.15.40:0/2738254182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe808066190 0x7fe8080707e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe8080b2380 tx=0x7fe80001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=271c9d52f2c79e69 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:14.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe7fffff640 1 -- 172.21.15.40:0/2738254182 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe800004230 con 0x7fe808066190 2023-10-16T08:39:14.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe7fffff640 1 -- 172.21.15.40:0/2738254182 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8000a7050 con 0x7fe808066190 2023-10-16T08:39:14.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe7fffff640 1 -- 172.21.15.40:0/2738254182 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe800003620 con 0x7fe808066190 2023-10-16T08:39:14.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80f957640 1 -- 172.21.15.40:0/2738254182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe808066190 msgr2=0x7fe8080707e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:14.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80f957640 1 --2- 172.21.15.40:0/2738254182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe808066190 0x7fe8080707e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fe8080b2380 tx=0x7fe80001c460 comp rx=0 tx=0).stop 2023-10-16T08:39:14.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80f957640 1 -- 172.21.15.40:0/2738254182 shutdown_connections 2023-10-16T08:39:14.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80f957640 1 --2- 172.21.15.40:0/2738254182 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe808066190 0x7fe8080707e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80f957640 1 --2- 172.21.15.40:0/2738254182 >> v2:172.21.15.86:3300/0 conn(0x7fe80815ad00 0x7fe8080cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80f957640 1 -- 172.21.15.40:0/2738254182 >> 172.21.15.40:0/2738254182 conn(0x7fe8080cc2d0 msgr2=0x7fe8080cc6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:14.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80f957640 1 -- 172.21.15.40:0/2738254182 shutdown_connections 2023-10-16T08:39:14.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80f957640 1 -- 172.21.15.40:0/2738254182 wait complete. 2023-10-16T08:39:14.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80f957640 1 Processor -- start 2023-10-16T08:39:14.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80f957640 1 -- start start 2023-10-16T08:39:14.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80f957640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe808066190 0x7fe808154010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:14.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80f957640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8080cb800 0x7fe80815ca80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:14.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80f957640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe80815ad00 0x7fe80815cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:14.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80f957640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8080ac980 con 0x7fe8080cb800 2023-10-16T08:39:14.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80f957640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8080acc60 con 0x7fe808066190 2023-10-16T08:39:14.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80f957640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8080acf40 con 0x7fe80815ad00 2023-10-16T08:39:14.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80cef2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8080cb800 0x7fe80815ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:14.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80cef2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8080cb800 0x7fe80815ca80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44668/0 (socket says 172.21.15.40:44668) 2023-10-16T08:39:14.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80cef2640 1 -- 172.21.15.40:0/279921514 learned_addr learned my addr 172.21.15.40:0/279921514 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:14.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80d6f3640 1 --2- 172.21.15.40:0/279921514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe808066190 0x7fe808154010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:14.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80def4640 1 --2- 172.21.15.40:0/279921514 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe80815ad00 0x7fe80815cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:14.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80cef2640 1 -- 172.21.15.40:0/279921514 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe80815ad00 msgr2=0x7fe80815cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:14.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80cef2640 1 --2- 172.21.15.40:0/279921514 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe80815ad00 0x7fe80815cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80cef2640 1 -- 172.21.15.40:0/279921514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe808066190 msgr2=0x7fe808154010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:14.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80cef2640 1 --2- 172.21.15.40:0/279921514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe808066190 0x7fe808154010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80cef2640 1 -- 172.21.15.40:0/279921514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe800009420 con 0x7fe8080cb800 2023-10-16T08:39:14.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.719+0000 7fe80cef2640 1 --2- 172.21.15.40:0/279921514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8080cb800 0x7fe80815ca80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800c4d0 tx=0x7fe7f800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:14.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.723+0000 7fe7fdffb640 1 -- 172.21.15.40:0/279921514 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8005cf0 con 0x7fe8080cb800 2023-10-16T08:39:14.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.723+0000 7fe7fdffb640 1 -- 172.21.15.40:0/279921514 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7f8005e50 con 0x7fe8080cb800 2023-10-16T08:39:14.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.723+0000 7fe7fdffb640 1 -- 172.21.15.40:0/279921514 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7f8007270 con 0x7fe8080cb800 2023-10-16T08:39:14.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.723+0000 7fe80f957640 1 -- 172.21.15.40:0/279921514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe80806ded0 con 0x7fe8080cb800 2023-10-16T08:39:14.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.723+0000 7fe80f957640 1 -- 172.21.15.40:0/279921514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe80815c670 con 0x7fe8080cb800 2023-10-16T08:39:14.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.723+0000 7fe80f957640 1 -- 172.21.15.40:0/279921514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe80804e9c0 con 0x7fe8080cb800 2023-10-16T08:39:14.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.723+0000 7fe7fdffb640 1 -- 172.21.15.40:0/279921514 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe7f80073d0 con 0x7fe8080cb800 2023-10-16T08:39:14.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.723+0000 7fe7fdffb640 1 --2- 172.21.15.40:0/279921514 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe7ec0664a0 0x7fe7ec068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:14.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.723+0000 7fe7fdffb640 1 -- 172.21.15.40:0/279921514 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7f80832a0 con 0x7fe8080cb800 2023-10-16T08:39:14.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.723+0000 7fe80d6f3640 1 --2- 172.21.15.40:0/279921514 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe7ec0664a0 0x7fe7ec068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:14.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.723+0000 7fe80d6f3640 1 --2- 172.21.15.40:0/279921514 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe7ec0664a0 0x7fe7ec068960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe804003fd0 tx=0x7fe8040039c0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:14.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.727+0000 7fe7fdffb640 1 -- 172.21.15.40:0/279921514 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fe7f8050860 con 0x7fe8080cb800 2023-10-16T08:39:14.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.883+0000 7fe80f957640 1 -- 172.21.15.40:0/279921514 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe8080ad3f0 con 0x7fe8080cb800 2023-10-16T08:39:14.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.883+0000 7fe7fdffb640 1 -- 172.21.15.40:0/279921514 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe7f8017020 con 0x7fe8080cb800 2023-10-16T08:39:14.889 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-16T08:39:14.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.887+0000 7fe80f957640 1 -- 172.21.15.40:0/279921514 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe7ec0664a0 msgr2=0x7fe7ec068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:14.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.887+0000 7fe80f957640 1 --2- 172.21.15.40:0/279921514 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe7ec0664a0 0x7fe7ec068960 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe804003fd0 tx=0x7fe8040039c0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.887+0000 7fe80f957640 1 -- 172.21.15.40:0/279921514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8080cb800 msgr2=0x7fe80815ca80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:14.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.887+0000 7fe80f957640 1 --2- 172.21.15.40:0/279921514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8080cb800 0x7fe80815ca80 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fe7f800c4d0 tx=0x7fe7f800c990 comp rx=0 tx=0).stop 2023-10-16T08:39:14.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.887+0000 7fe80f957640 1 -- 172.21.15.40:0/279921514 shutdown_connections 2023-10-16T08:39:14.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.887+0000 7fe80f957640 1 --2- 172.21.15.40:0/279921514 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe80815ad00 0x7fe80815cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.887+0000 7fe80f957640 1 --2- 172.21.15.40:0/279921514 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe7ec0664a0 0x7fe7ec068960 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.887+0000 7fe80f957640 1 --2- 172.21.15.40:0/279921514 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8080cb800 0x7fe80815ca80 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.887+0000 7fe80f957640 1 --2- 172.21.15.40:0/279921514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe808066190 0x7fe808154010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:14.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.887+0000 7fe80f957640 1 -- 172.21.15.40:0/279921514 >> 172.21.15.40:0/279921514 conn(0x7fe8080cc2d0 msgr2=0x7fe808159d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:14.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.887+0000 7fe80f957640 1 -- 172.21.15.40:0/279921514 shutdown_connections 2023-10-16T08:39:14.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:14.887+0000 7fe80f957640 1 -- 172.21.15.40:0/279921514 wait complete. 2023-10-16T08:39:14.908 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:39:15.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.035+0000 7f0f965d4640 1 Processor -- start 2023-10-16T08:39:15.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.035+0000 7f0f965d4640 1 -- start start 2023-10-16T08:39:15.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.035+0000 7f0f965d4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f9015cdc0 0x7f0f900d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:15.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.035+0000 7f0f965d4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0f900d3310 0x7f0f900b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:15.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.035+0000 7f0f965d4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f900d1870 con 0x7f0f900b8590 2023-10-16T08:39:15.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.035+0000 7f0f965d4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f900d1b50 con 0x7f0f900d3310 2023-10-16T08:39:15.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.035+0000 7f0f965d4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f900d1e30 con 0x7f0f9015cdc0 2023-10-16T08:39:15.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.035+0000 7f0f94b71640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0f900d3310 0x7f0f900b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:15.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.035+0000 7f0f8f7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f0f900b8590 0x7f0f900d2c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60852/0 (socket says 172.21.15.40:60852) 2023-10-16T08:39:15.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.035+0000 7f0f94b71640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0f900d3310 0x7f0f900b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44682/0 (socket says 172.21.15.40:44682) 2023-10-16T08:39:15.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.035+0000 7f0f8f7fe640 1 -- 172.21.15.40:0/4032208178 learned_addr learned my addr 172.21.15.40:0/4032208178 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:15.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.035+0000 7f0f94b71640 1 -- 172.21.15.40:0/4032208178 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f9015cdc0 msgr2=0x7f0f900d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:15.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.035+0000 7f0f94b71640 1 --2- 172.21.15.40:0/4032208178 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f9015cdc0 0x7f0f900d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.035+0000 7f0f94b71640 1 -- 172.21.15.40:0/4032208178 >> v1:172.21.15.40:6789/0 conn(0x7f0f900b8590 legacy=0x7f0f900d2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:15.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.035+0000 7f0f94b71640 1 -- 172.21.15.40:0/4032208178 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f900d2110 con 0x7f0f900d3310 2023-10-16T08:39:15.043 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.035+0000 7f0f8effd640 1 -- 172.21.15.40:0/4032208178 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3455394316 0 0) 0x7f0f900d1870 con 0x7f0f900b8590 2023-10-16T08:39:15.044 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f94b71640 1 --2- 172.21.15.40:0/4032208178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f900d3310 0x7f0f900b5da0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0f880097c0 tx=0x7f0f8801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e07a892b97b6af4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:15.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f8effd640 1 -- 172.21.15.40:0/4032208178 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f880a7070 con 0x7f0f900d3310 2023-10-16T08:39:15.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f8effd640 1 -- 172.21.15.40:0/4032208178 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f880ab430 con 0x7f0f900d3310 2023-10-16T08:39:15.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f8effd640 1 -- 172.21.15.40:0/4032208178 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f880ab7c0 con 0x7f0f900d3310 2023-10-16T08:39:15.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f965d4640 1 -- 172.21.15.40:0/4032208178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f900d3310 msgr2=0x7f0f900b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:15.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f965d4640 1 --2- 172.21.15.40:0/4032208178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f900d3310 0x7f0f900b5da0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0f880097c0 tx=0x7f0f8801c450 comp rx=0 tx=0).stop 2023-10-16T08:39:15.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f965d4640 1 -- 172.21.15.40:0/4032208178 shutdown_connections 2023-10-16T08:39:15.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f965d4640 1 --2- 172.21.15.40:0/4032208178 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f900d3310 0x7f0f900b5da0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f965d4640 1 --2- 172.21.15.40:0/4032208178 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f9015cdc0 0x7f0f900d1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f965d4640 1 -- 172.21.15.40:0/4032208178 >> 172.21.15.40:0/4032208178 conn(0x7f0f9006a290 msgr2=0x7f0f9006a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:15.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f965d4640 1 -- 172.21.15.40:0/4032208178 shutdown_connections 2023-10-16T08:39:15.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f965d4640 1 -- 172.21.15.40:0/4032208178 wait complete. 2023-10-16T08:39:15.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f965d4640 1 Processor -- start 2023-10-16T08:39:15.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f965d4640 1 -- start start 2023-10-16T08:39:15.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f965d4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f900b8590 0x7f0f900c8b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:15.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f965d4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f900d3310 0x7f0f9015fe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:15.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f965d4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9015cdc0 0x7f0f900c6e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:15.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f965d4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f9015a500 con 0x7f0f9015cdc0 2023-10-16T08:39:15.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f965d4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f901596a0 con 0x7f0f900b8590 2023-10-16T08:39:15.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f965d4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f90159980 con 0x7f0f900d3310 2023-10-16T08:39:15.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f94b71640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9015cdc0 0x7f0f900c6e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:15.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f94b71640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9015cdc0 0x7f0f900c6e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44688/0 (socket says 172.21.15.40:44688) 2023-10-16T08:39:15.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f94b71640 1 -- 172.21.15.40:0/3275456131 learned_addr learned my addr 172.21.15.40:0/3275456131 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:15.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f8f7fe640 1 --2- 172.21.15.40:0/3275456131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f900d3310 0x7f0f9015fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:15.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f8ffff640 1 --2- 172.21.15.40:0/3275456131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f900b8590 0x7f0f900c8b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:15.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f94b71640 1 -- 172.21.15.40:0/3275456131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f900d3310 msgr2=0x7f0f9015fe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:15.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f94b71640 1 --2- 172.21.15.40:0/3275456131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f900d3310 0x7f0f9015fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f94b71640 1 -- 172.21.15.40:0/3275456131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f900b8590 msgr2=0x7f0f900c8b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:15.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f94b71640 1 --2- 172.21.15.40:0/3275456131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f900b8590 0x7f0f900c8b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f94b71640 1 -- 172.21.15.40:0/3275456131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f88009420 con 0x7f0f9015cdc0 2023-10-16T08:39:15.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f94b71640 1 --2- 172.21.15.40:0/3275456131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9015cdc0 0x7f0f900c6e20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0f880ad040 tx=0x7f0f880191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f8cff9640 1 -- 172.21.15.40:0/3275456131 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f880bc620 con 0x7f0f9015cdc0 2023-10-16T08:39:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f8cff9640 1 -- 172.21.15.40:0/3275456131 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f880bc780 con 0x7f0f9015cdc0 2023-10-16T08:39:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f965d4640 1 -- 172.21.15.40:0/3275456131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f90159c60 con 0x7f0f9015cdc0 2023-10-16T08:39:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f8cff9640 1 -- 172.21.15.40:0/3275456131 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f880bcb30 con 0x7f0f9015cdc0 2023-10-16T08:39:15.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.039+0000 7f0f965d4640 1 -- 172.21.15.40:0/3275456131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f9015a220 con 0x7f0f9015cdc0 2023-10-16T08:39:15.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.043+0000 7f0f965d4640 1 -- 172.21.15.40:0/3275456131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f900c76a0 con 0x7f0f9015cdc0 2023-10-16T08:39:15.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.043+0000 7f0f8cff9640 1 -- 172.21.15.40:0/3275456131 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0f880c5450 con 0x7f0f9015cdc0 2023-10-16T08:39:15.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.043+0000 7f0f8cff9640 1 --2- 172.21.15.40:0/3275456131 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0f7c0664a0 0x7f0f7c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:15.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.043+0000 7f0f8cff9640 1 -- 172.21.15.40:0/3275456131 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0f880b8070 con 0x7f0f9015cdc0 2023-10-16T08:39:15.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.043+0000 7f0f8ffff640 1 --2- 172.21.15.40:0/3275456131 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0f7c0664a0 0x7f0f7c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:15.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.043+0000 7f0f8ffff640 1 --2- 172.21.15.40:0/3275456131 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0f7c0664a0 0x7f0f7c068960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0f9015da20 tx=0x7f0f80007310 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:15.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.047+0000 7f0f8cff9640 1 -- 172.21.15.40:0/3275456131 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f0f900c76a0 con 0x7f0f9015cdc0 2023-10-16T08:39:15.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.203+0000 7f0f965d4640 1 -- 172.21.15.40:0/3275456131 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0f900c76a0 con 0x7f0f9015cdc0 2023-10-16T08:39:15.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.203+0000 7f0f8cff9640 1 -- 172.21.15.40:0/3275456131 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0f900c76a0 con 0x7f0f9015cdc0 2023-10-16T08:39:15.209 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-16T08:39:15.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.207+0000 7f0f965d4640 1 -- 172.21.15.40:0/3275456131 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0f7c0664a0 msgr2=0x7f0f7c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:15.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.207+0000 7f0f965d4640 1 --2- 172.21.15.40:0/3275456131 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0f7c0664a0 0x7f0f7c068960 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0f9015da20 tx=0x7f0f80007310 comp rx=0 tx=0).stop 2023-10-16T08:39:15.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.207+0000 7f0f965d4640 1 -- 172.21.15.40:0/3275456131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9015cdc0 msgr2=0x7f0f900c6e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:15.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.207+0000 7f0f965d4640 1 --2- 172.21.15.40:0/3275456131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9015cdc0 0x7f0f900c6e20 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f0f880ad040 tx=0x7f0f880191f0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.207+0000 7f0f965d4640 1 -- 172.21.15.40:0/3275456131 shutdown_connections 2023-10-16T08:39:15.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.207+0000 7f0f965d4640 1 --2- 172.21.15.40:0/3275456131 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0f7c0664a0 0x7f0f7c068960 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.207+0000 7f0f965d4640 1 --2- 172.21.15.40:0/3275456131 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0f9015cdc0 0x7f0f900c6e20 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.207+0000 7f0f965d4640 1 --2- 172.21.15.40:0/3275456131 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0f900d3310 0x7f0f9015fe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.207+0000 7f0f965d4640 1 --2- 172.21.15.40:0/3275456131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0f900b8590 0x7f0f900c8b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.207+0000 7f0f965d4640 1 -- 172.21.15.40:0/3275456131 >> 172.21.15.40:0/3275456131 conn(0x7f0f9006a290 msgr2=0x7f0f900d2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:15.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.207+0000 7f0f965d4640 1 -- 172.21.15.40:0/3275456131 shutdown_connections 2023-10-16T08:39:15.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.207+0000 7f0f965d4640 1 -- 172.21.15.40:0/3275456131 wait complete. 2023-10-16T08:39:15.228 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:39:15.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.351+0000 7f02a3dd5640 1 Processor -- start 2023-10-16T08:39:15.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.351+0000 7f02a3dd5640 1 -- start start 2023-10-16T08:39:15.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.351+0000 7f02a3dd5640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f029c15abd0 0x7f029c0b2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:15.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.351+0000 7f02a3dd5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f029c0b2ac0 0x7f029c0af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:15.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.351+0000 7f02a3dd5640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f029c0714d0 con 0x7f029c0afd20 2023-10-16T08:39:15.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.351+0000 7f02a3dd5640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f029c0b2f20 con 0x7f029c0b2ac0 2023-10-16T08:39:15.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.351+0000 7f02a3dd5640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f029c0b0190 con 0x7f029c15abd0 2023-10-16T08:39:15.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.351+0000 7f02a2372640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f029c0afd20 0x7f029c06ef20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60856/0 (socket says 172.21.15.40:60856) 2023-10-16T08:39:15.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.351+0000 7f02a2372640 1 -- 172.21.15.40:0/1713560777 learned_addr learned my addr 172.21.15.40:0/1713560777 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:15.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.351+0000 7f02a1370640 1 --2- 172.21.15.40:0/1713560777 >> v2:172.21.15.40:3300/0 conn(0x7f029c0b2ac0 0x7f029c0af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:15.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.351+0000 7f02a1b71640 1 --2- 172.21.15.40:0/1713560777 >> v2:172.21.15.86:3300/0 conn(0x7f029c15abd0 0x7f029c0b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:15.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.351+0000 7f02a1370640 1 -- 172.21.15.40:0/1713560777 >> v2:172.21.15.86:3300/0 conn(0x7f029c15abd0 msgr2=0x7f029c0b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:15.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.351+0000 7f02a1370640 1 --2- 172.21.15.40:0/1713560777 >> v2:172.21.15.86:3300/0 conn(0x7f029c15abd0 0x7f029c0b2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.351+0000 7f02a1370640 1 -- 172.21.15.40:0/1713560777 >> v1:172.21.15.40:6789/0 conn(0x7f029c0afd20 legacy=0x7f029c06ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:15.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.351+0000 7f02a1370640 1 -- 172.21.15.40:0/1713560777 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f029c0722a0 con 0x7f029c0b2ac0 2023-10-16T08:39:15.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a1370640 1 --2- 172.21.15.40:0/1713560777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f029c0b2ac0 0x7f029c0af7e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f029c0b31a0 tx=0x7f029801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a06278e040544274 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:15.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a0b6f640 1 -- 172.21.15.40:0/1713560777 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02980ab600 con 0x7f029c0b2ac0 2023-10-16T08:39:15.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a0b6f640 1 -- 172.21.15.40:0/1713560777 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02980a7050 con 0x7f029c0b2ac0 2023-10-16T08:39:15.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a0b6f640 1 -- 172.21.15.40:0/1713560777 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02980ab980 con 0x7f029c0b2ac0 2023-10-16T08:39:15.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a3dd5640 1 -- 172.21.15.40:0/1713560777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f029c0b2ac0 msgr2=0x7f029c0af7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:15.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a3dd5640 1 --2- 172.21.15.40:0/1713560777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f029c0b2ac0 0x7f029c0af7e0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f029c0b31a0 tx=0x7f029801c3d0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a3dd5640 1 -- 172.21.15.40:0/1713560777 shutdown_connections 2023-10-16T08:39:15.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a3dd5640 1 --2- 172.21.15.40:0/1713560777 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f029c0b2ac0 0x7f029c0af7e0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a3dd5640 1 --2- 172.21.15.40:0/1713560777 >> v2:172.21.15.86:3300/0 conn(0x7f029c15abd0 0x7f029c0b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a3dd5640 1 -- 172.21.15.40:0/1713560777 >> 172.21.15.40:0/1713560777 conn(0x7f029c0cd3c0 msgr2=0x7f029c0cd7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:15.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a3dd5640 1 -- 172.21.15.40:0/1713560777 shutdown_connections 2023-10-16T08:39:15.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a3dd5640 1 -- 172.21.15.40:0/1713560777 wait complete. 2023-10-16T08:39:15.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a3dd5640 1 Processor -- start 2023-10-16T08:39:15.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a3dd5640 1 -- start start 2023-10-16T08:39:15.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a3dd5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f029c0afd20 0x7f029c0c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:15.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a3dd5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f029c0b2ac0 0x7f029c0b3460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:15.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a3dd5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f029c15abd0 0x7f029c0b39a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:15.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a3dd5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c0c8100 con 0x7f029c0afd20 2023-10-16T08:39:15.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a3dd5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f029c0ca810 con 0x7f029c0b2ac0 2023-10-16T08:39:15.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a3dd5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f029c0ca530 con 0x7f029c15abd0 2023-10-16T08:39:15.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a1b71640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f029c0afd20 0x7f029c0c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:15.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a1b71640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f029c0afd20 0x7f029c0c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44706/0 (socket says 172.21.15.40:44706) 2023-10-16T08:39:15.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.355+0000 7f02a1b71640 1 -- 172.21.15.40:0/3161578158 learned_addr learned my addr 172.21.15.40:0/3161578158 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:15.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.359+0000 7f02a2372640 1 --2- 172.21.15.40:0/3161578158 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f029c15abd0 0x7f029c0b39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.359+0000 7f02a1370640 1 --2- 172.21.15.40:0/3161578158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f029c0b2ac0 0x7f029c0b3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.359+0000 7f02a1b71640 1 -- 172.21.15.40:0/3161578158 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f029c15abd0 msgr2=0x7f029c0b39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.359+0000 7f02a1b71640 1 --2- 172.21.15.40:0/3161578158 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f029c15abd0 0x7f029c0b39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.359+0000 7f02a1b71640 1 -- 172.21.15.40:0/3161578158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f029c0b2ac0 msgr2=0x7f029c0b3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:15.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.359+0000 7f02a1b71640 1 --2- 172.21.15.40:0/3161578158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f029c0b2ac0 0x7f029c0b3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.359+0000 7f02a1b71640 1 -- 172.21.15.40:0/3161578158 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0298009420 con 0x7f029c0afd20 2023-10-16T08:39:15.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.359+0000 7f02a1b71640 1 --2- 172.21.15.40:0/3161578158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f029c0afd20 0x7f029c0c8590 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f028c00d7d0 tx=0x7f028c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:15.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.359+0000 7f02927fc640 1 -- 172.21.15.40:0/3161578158 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c00b6c0 con 0x7f029c0afd20 2023-10-16T08:39:15.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.359+0000 7f02927fc640 1 -- 172.21.15.40:0/3161578158 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f028c00b820 con 0x7f029c0afd20 2023-10-16T08:39:15.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.359+0000 7f02a3dd5640 1 -- 172.21.15.40:0/3161578158 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f029c0c7ab0 con 0x7f029c0afd20 2023-10-16T08:39:15.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.359+0000 7f02927fc640 1 -- 172.21.15.40:0/3161578158 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f028c00bbd0 con 0x7f029c0afd20 2023-10-16T08:39:15.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.359+0000 7f02a3dd5640 1 -- 172.21.15.40:0/3161578158 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f029c157610 con 0x7f029c0afd20 2023-10-16T08:39:15.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.359+0000 7f02a3dd5640 1 -- 172.21.15.40:0/3161578158 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f029c04e9c0 con 0x7f029c0afd20 2023-10-16T08:39:15.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.359+0000 7f02927fc640 1 -- 172.21.15.40:0/3161578158 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f028c00bd30 con 0x7f029c0afd20 2023-10-16T08:39:15.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.359+0000 7f02927fc640 1 --2- 172.21.15.40:0/3161578158 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f02800664a0 0x7f0280068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:15.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.359+0000 7f02927fc640 1 -- 172.21.15.40:0/3161578158 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f028c084380 con 0x7f029c0afd20 2023-10-16T08:39:15.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.359+0000 7f02a1370640 1 --2- 172.21.15.40:0/3161578158 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f02800664a0 0x7f0280068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:15.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.359+0000 7f02a1370640 1 --2- 172.21.15.40:0/3161578158 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f02800664a0 0x7f0280068960 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f02980ad000 tx=0x7f0298008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:15.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.363+0000 7f02927fc640 1 -- 172.21.15.40:0/3161578158 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f028c051800 con 0x7f029c0afd20 2023-10-16T08:39:15.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.515+0000 7f02a3dd5640 1 -- 172.21.15.40:0/3161578158 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f029c157a20 con 0x7f029c0afd20 2023-10-16T08:39:15.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.515+0000 7f02927fc640 1 -- 172.21.15.40:0/3161578158 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f028c054e20 con 0x7f029c0afd20 2023-10-16T08:39:15.520 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-16T08:39:15.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.515+0000 7f02a3dd5640 1 -- 172.21.15.40:0/3161578158 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f02800664a0 msgr2=0x7f0280068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:15.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.515+0000 7f02a3dd5640 1 --2- 172.21.15.40:0/3161578158 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f02800664a0 0x7f0280068960 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f02980ad000 tx=0x7f0298008040 comp rx=0 tx=0).stop 2023-10-16T08:39:15.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.515+0000 7f02a3dd5640 1 -- 172.21.15.40:0/3161578158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f029c0afd20 msgr2=0x7f029c0c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:15.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.515+0000 7f02a3dd5640 1 --2- 172.21.15.40:0/3161578158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f029c0afd20 0x7f029c0c8590 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f028c00d7d0 tx=0x7f028c00dc90 comp rx=0 tx=0).stop 2023-10-16T08:39:15.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.515+0000 7f02a3dd5640 1 -- 172.21.15.40:0/3161578158 shutdown_connections 2023-10-16T08:39:15.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.515+0000 7f02a3dd5640 1 --2- 172.21.15.40:0/3161578158 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f029c15abd0 0x7f029c0b39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.515+0000 7f02a3dd5640 1 --2- 172.21.15.40:0/3161578158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f029c0b2ac0 0x7f029c0b3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.515+0000 7f02a3dd5640 1 --2- 172.21.15.40:0/3161578158 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f02800664a0 0x7f0280068960 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.515+0000 7f02a3dd5640 1 --2- 172.21.15.40:0/3161578158 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f029c0afd20 0x7f029c0c8590 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.515+0000 7f02a3dd5640 1 -- 172.21.15.40:0/3161578158 >> 172.21.15.40:0/3161578158 conn(0x7f029c0cd3c0 msgr2=0x7f029c06ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:15.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.515+0000 7f02a3dd5640 1 -- 172.21.15.40:0/3161578158 shutdown_connections 2023-10-16T08:39:15.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.515+0000 7f02a3dd5640 1 -- 172.21.15.40:0/3161578158 wait complete. 2023-10-16T08:39:15.543 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_mchangir_ceph_6d9e81bfe885e1cd43cce3f41c9616ed0dce2cf3/qa/tasks/cephfs_test_runner.py", line 166, in task module_suites.append(decorating_loader.loadTestsFromName(mod_name)) File "/home/teuthworker/src/github.com_mchangir_ceph_6d9e81bfe885e1cd43cce3f41c9616ed0dce2cf3/qa/tasks/cephfs_test_runner.py", line 36, in loadTestsFromName result = super(DecoratingLoader, self).loadTestsFromName(name, module) File "/usr/lib/python3.8/unittest/loader.py", line 154, in loadTestsFromName module = __import__(module_name) File "/home/teuthworker/src/github.com_mchangir_ceph_6d9e81bfe885e1cd43cce3f41c9616ed0dce2cf3/qa/tasks/cephfs/test_snap_schedules.py", line 586 self.fs_snap_schedule_cmd('add', path='.', snap_schedule='1m', '--subvol', 'sv1') ^ SyntaxError: positional argument follows keyword argument 2023-10-16T08:39:15.609 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=97de75c98ad146b3b8c10d9eaa391c68 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_8cdab074dcca9a68965bc5a50e9c30b691949723/teuthology/run_tasks.py", line 109, in run_tasks manager.__enter__() File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__ return next(self.gen) File "/home/teuthworker/src/github.com_mchangir_ceph_6d9e81bfe885e1cd43cce3f41c9616ed0dce2cf3/qa/tasks/cephfs_test_runner.py", line 166, in task module_suites.append(decorating_loader.loadTestsFromName(mod_name)) File "/home/teuthworker/src/github.com_mchangir_ceph_6d9e81bfe885e1cd43cce3f41c9616ed0dce2cf3/qa/tasks/cephfs_test_runner.py", line 36, in loadTestsFromName result = super(DecoratingLoader, self).loadTestsFromName(name, module) File "/usr/lib/python3.8/unittest/loader.py", line 154, in loadTestsFromName module = __import__(module_name) File "/home/teuthworker/src/github.com_mchangir_ceph_6d9e81bfe885e1cd43cce3f41c9616ed0dce2cf3/qa/tasks/cephfs/test_snap_schedules.py", line 586 self.fs_snap_schedule_cmd('add', path='.', snap_schedule='1m', '--subvol', 'sv1') ^ SyntaxError: positional argument follows keyword argument 2023-10-16T08:39:15.611 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-16T08:39:15.621 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-10-16T08:39:15.668 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-10-16T08:39:15.669 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:39:15.669 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:39:15.683 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:39:15.684 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:39:15.732 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:39:15.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac441d640 1 Processor -- start 2023-10-16T08:39:15.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac441d640 1 -- start start 2023-10-16T08:39:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac441d640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f8abc0adec0 0x7f8abc06bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac441d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8abc0adca0 con 0x7f8abc15b040 2023-10-16T08:39:15.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac441d640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8abc0c5840 con 0x7f8abc0cae50 2023-10-16T08:39:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac441d640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8abc0cf890 con 0x7f8abc0adec0 2023-10-16T08:39:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac21b9640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f8abc15b040 0x7f8abc0ca740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60872/0 (socket says 172.21.15.40:60872) 2023-10-16T08:39:15.904 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac21b9640 1 -- 172.21.15.40:0/1124473862 learned_addr learned my addr 172.21.15.40:0/1124473862 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac29ba640 1 --2- 172.21.15.40:0/1124473862 >> v2:172.21.15.86:3300/0 conn(0x7f8abc0adec0 0x7f8abc06bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac11b7640 1 -- 172.21.15.40:0/1124473862 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2414619634 0 0) 0x7f8abc0adca0 con 0x7f8abc15b040 2023-10-16T08:39:15.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac11b7640 1 -- 172.21.15.40:0/1124473862 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8aa8003530 con 0x7f8abc15b040 2023-10-16T08:39:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac11b7640 1 -- 172.21.15.40:0/1124473862 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3556086315 0 0) 0x7f8aa8003530 con 0x7f8abc15b040 2023-10-16T08:39:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac11b7640 1 -- 172.21.15.40:0/1124473862 >> v2:172.21.15.86:3300/0 conn(0x7f8abc0adec0 msgr2=0x7f8abc06bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:15.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac11b7640 1 --2- 172.21.15.40:0/1124473862 >> v2:172.21.15.86:3300/0 conn(0x7f8abc0adec0 0x7f8abc06bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac11b7640 1 -- 172.21.15.40:0/1124473862 >> v1:172.21.15.86:6789/0 conn(0x7f8abc0cae50 legacy=0x7f8abc06b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:15.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac11b7640 1 -- 172.21.15.40:0/1124473862 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc0cf5b0 con 0x7f8abc15b040 2023-10-16T08:39:15.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac11b7640 1 -- 172.21.15.40:0/1124473862 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f8ab8001680 con 0x7f8abc15b040 2023-10-16T08:39:15.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac11b7640 1 -- 172.21.15.40:0/1124473862 >> v1:172.21.15.40:6789/0 conn(0x7f8abc15b040 legacy=0x7f8abc0ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:15.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac11b7640 1 --2- 172.21.15.40:0/1124473862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa80039f0 0x7f8aa8003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:15.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac11b7640 1 --2- 172.21.15.40:0/1124473862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8aa80047d0 0x7f8aa8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:15.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac11b7640 1 --2- 172.21.15.40:0/1124473862 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8aa8005290 0x7f8aa800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:15.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac11b7640 1 -- 172.21.15.40:0/1124473862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc0cf890 con 0x7f8aa80047d0 2023-10-16T08:39:15.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac11b7640 1 -- 172.21.15.40:0/1124473862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa800c1d0 con 0x7f8aa80039f0 2023-10-16T08:39:15.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac11b7640 1 -- 172.21.15.40:0/1124473862 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa800c390 con 0x7f8aa8005290 2023-10-16T08:39:15.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac29ba640 1 --2- 172.21.15.40:0/1124473862 >> v2:172.21.15.86:3300/0 conn(0x7f8abc0adec0 0x7f8abc06bec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:39:15.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac19b8640 1 --2- 172.21.15.40:0/1124473862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8aa80047d0 0x7f8aa8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:15.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac21b9640 1 --2- 172.21.15.40:0/1124473862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa80039f0 0x7f8aa8003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:15.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac29ba640 1 --2- 172.21.15.40:0/1124473862 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8aa8005290 0x7f8aa800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:15.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac19b8640 1 -- 172.21.15.40:0/1124473862 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8aa8005290 msgr2=0x7f8aa800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac19b8640 1 --2- 172.21.15.40:0/1124473862 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8aa8005290 0x7f8aa800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac19b8640 1 -- 172.21.15.40:0/1124473862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa80039f0 msgr2=0x7f8aa8003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac19b8640 1 --2- 172.21.15.40:0/1124473862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa80039f0 0x7f8aa8003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac19b8640 1 -- 172.21.15.40:0/1124473862 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa8003530 con 0x7f8aa80047d0 2023-10-16T08:39:15.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac29ba640 1 --2- 172.21.15.40:0/1124473862 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8aa8005290 0x7f8aa800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:15.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac21b9640 1 --2- 172.21.15.40:0/1124473862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8aa80039f0 0x7f8aa8003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:15.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.895+0000 7f8ac29ba640 1 -- 172.21.15.40:0/1124473862 reap_dead start 2023-10-16T08:39:15.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.899+0000 7f8ac19b8640 1 --2- 172.21.15.40:0/1124473862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8aa80047d0 0x7f8aa8004c50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8aac001780 tx=0x7f8aac01c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20bbfa0827490330 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:15.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.899+0000 7f8ac11b7640 1 -- 172.21.15.40:0/1124473862 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac0052d0 con 0x7f8aa80047d0 2023-10-16T08:39:15.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.899+0000 7f8ac11b7640 1 -- 172.21.15.40:0/1124473862 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8aac0a7050 con 0x7f8aa80047d0 2023-10-16T08:39:15.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.899+0000 7f8ac11b7640 1 -- 172.21.15.40:0/1124473862 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8aac003600 con 0x7f8aa80047d0 2023-10-16T08:39:15.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.899+0000 7f8ac441d640 1 -- 172.21.15.40:0/1124473862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8aa80047d0 msgr2=0x7f8aa8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:15.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.899+0000 7f8ac441d640 1 --2- 172.21.15.40:0/1124473862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8aa80047d0 0x7f8aa8004c50 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8aac001780 tx=0x7f8aac01c3b0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.899+0000 7f8ac441d640 1 -- 172.21.15.40:0/1124473862 shutdown_connections 2023-10-16T08:39:15.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.899+0000 7f8ac441d640 1 --2- 172.21.15.40:0/1124473862 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8aa80047d0 0x7f8aa8004c50 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.899+0000 7f8ac441d640 1 -- 172.21.15.40:0/1124473862 >> 172.21.15.40:0/1124473862 conn(0x7f8abc0ce960 msgr2=0x7f8abc0ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:15.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.899+0000 7f8ac441d640 1 -- 172.21.15.40:0/1124473862 shutdown_connections 2023-10-16T08:39:15.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.899+0000 7f8ac441d640 1 -- 172.21.15.40:0/1124473862 wait complete. 2023-10-16T08:39:15.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.899+0000 7f8ac441d640 1 Processor -- start 2023-10-16T08:39:15.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.899+0000 7f8ac441d640 1 -- start start 2023-10-16T08:39:15.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac441d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8aa80047d0 0x7f8abc0c5a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:15.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac441d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8abc155170 0x7f8abc158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:15.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac441d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8abc1593a0 0x7f8abc0acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:15.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac441d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc0c3e10 con 0x7f8aa80047d0 2023-10-16T08:39:15.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac441d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc159820 con 0x7f8abc155170 2023-10-16T08:39:15.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac441d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc0cc6d0 con 0x7f8abc1593a0 2023-10-16T08:39:15.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac21b9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8aa80047d0 0x7f8abc0c5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:15.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac21b9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8aa80047d0 0x7f8abc0c5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44714/0 (socket says 172.21.15.40:44714) 2023-10-16T08:39:15.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac21b9640 1 -- 172.21.15.40:0/4286281651 learned_addr learned my addr 172.21.15.40:0/4286281651 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:15.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac29ba640 1 --2- 172.21.15.40:0/4286281651 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8abc1593a0 0x7f8abc0acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:15.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac19b8640 1 --2- 172.21.15.40:0/4286281651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8abc155170 0x7f8abc158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac21b9640 1 -- 172.21.15.40:0/4286281651 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8abc1593a0 msgr2=0x7f8abc0acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac21b9640 1 --2- 172.21.15.40:0/4286281651 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8abc1593a0 0x7f8abc0acaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac21b9640 1 -- 172.21.15.40:0/4286281651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8abc155170 msgr2=0x7f8abc158e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:15.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac21b9640 1 --2- 172.21.15.40:0/4286281651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8abc155170 0x7f8abc158e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:15.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac21b9640 1 -- 172.21.15.40:0/4286281651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc0cc9b0 con 0x7f8aa80047d0 2023-10-16T08:39:15.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac21b9640 1 --2- 172.21.15.40:0/4286281651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8aa80047d0 0x7f8abc0c5a30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800e4d0 tx=0x7f8ab800e990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:15.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ab2ffd640 1 -- 172.21.15.40:0/4286281651 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8012650 con 0x7f8aa80047d0 2023-10-16T08:39:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ab2ffd640 1 -- 172.21.15.40:0/4286281651 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab80127b0 con 0x7f8aa80047d0 2023-10-16T08:39:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac441d640 1 -- 172.21.15.40:0/4286281651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8abc0ccc90 con 0x7f8aa80047d0 2023-10-16T08:39:15.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ab2ffd640 1 -- 172.21.15.40:0/4286281651 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8012b60 con 0x7f8aa80047d0 2023-10-16T08:39:15.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac441d640 1 -- 172.21.15.40:0/4286281651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8abc0cd250 con 0x7f8aa80047d0 2023-10-16T08:39:15.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac441d640 1 -- 172.21.15.40:0/4286281651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8abc04e9c0 con 0x7f8aa80047d0 2023-10-16T08:39:15.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ab2ffd640 1 -- 172.21.15.40:0/4286281651 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f8ab8012cc0 con 0x7f8aa80047d0 2023-10-16T08:39:15.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ab2ffd640 1 --2- 172.21.15.40:0/4286281651 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f8aa00664d0 0x7f8aa0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ab2ffd640 1 -- 172.21.15.40:0/4286281651 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8ab80838f0 con 0x7f8aa80047d0 2023-10-16T08:39:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.903+0000 7f8ac19b8640 1 --2- 172.21.15.40:0/4286281651 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f8aa00664d0 0x7f8aa0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.907+0000 7f8ac19b8640 1 --2- 172.21.15.40:0/4286281651 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f8aa00664d0 0x7f8aa0068990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8aac001750 tx=0x7f8aac0011a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:15.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:15.907+0000 7f8ab2ffd640 1 -- 172.21.15.40:0/4286281651 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f8ab8050d70 con 0x7f8aa80047d0 2023-10-16T08:39:16.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.063+0000 7f8ac441d640 1 -- 172.21.15.40:0/4286281651 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8abc06c9e0 con 0x7f8aa80047d0 2023-10-16T08:39:16.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.067+0000 7f8ab2ffd640 1 -- 172.21.15.40:0/4286281651 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f8ab8054390 con 0x7f8aa80047d0 2023-10-16T08:39:16.072 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:39:16.072 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:46.611098+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:39:16.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.067+0000 7f8ac441d640 1 -- 172.21.15.40:0/4286281651 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f8aa00664d0 msgr2=0x7f8aa0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:16.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.067+0000 7f8ac441d640 1 --2- 172.21.15.40:0/4286281651 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f8aa00664d0 0x7f8aa0068990 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f8aac001750 tx=0x7f8aac0011a0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.067+0000 7f8ac441d640 1 -- 172.21.15.40:0/4286281651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8aa80047d0 msgr2=0x7f8abc0c5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:16.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.067+0000 7f8ac441d640 1 --2- 172.21.15.40:0/4286281651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8aa80047d0 0x7f8abc0c5a30 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800e4d0 tx=0x7f8ab800e990 comp rx=0 tx=0).stop 2023-10-16T08:39:16.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.067+0000 7f8ac441d640 1 -- 172.21.15.40:0/4286281651 shutdown_connections 2023-10-16T08:39:16.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.067+0000 7f8ac441d640 1 --2- 172.21.15.40:0/4286281651 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f8abc1593a0 0x7f8abc0acaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.067+0000 7f8ac441d640 1 --2- 172.21.15.40:0/4286281651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8abc155170 0x7f8abc158e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.067+0000 7f8ac441d640 1 --2- 172.21.15.40:0/4286281651 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f8aa00664d0 0x7f8aa0068990 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.067+0000 7f8ac441d640 1 --2- 172.21.15.40:0/4286281651 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f8aa80047d0 0x7f8abc0c5a30 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.067+0000 7f8ac441d640 1 -- 172.21.15.40:0/4286281651 >> 172.21.15.40:0/4286281651 conn(0x7f8abc0ce960 msgr2=0x7f8abc0b3870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:16.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.067+0000 7f8ac441d640 1 -- 172.21.15.40:0/4286281651 shutdown_connections 2023-10-16T08:39:16.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.067+0000 7f8ac441d640 1 -- 172.21.15.40:0/4286281651 wait complete. 2023-10-16T08:39:16.089 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:38:33.014668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:38:43.055215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:38:43.971666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:39:16.090 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:39:16.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.207+0000 7f1056e7d640 1 Processor -- start 2023-10-16T08:39:16.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f1056e7d640 1 -- start start 2023-10-16T08:39:16.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f1056e7d640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1050159dd0 0x7f10500c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f1056e7d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10500adec0 0x7f10500c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f1056e7d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105015b490 con 0x7f105006b800 2023-10-16T08:39:16.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f1056e7d640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105006cd70 con 0x7f1050159dd0 2023-10-16T08:39:16.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f1056e7d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105006d230 con 0x7f10500adec0 2023-10-16T08:39:16.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f105541a640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f105006b800 0x7f10500cc460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60884/0 (socket says 172.21.15.40:60884) 2023-10-16T08:39:16.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f105541a640 1 -- 172.21.15.40:0/1019410796 learned_addr learned my addr 172.21.15.40:0/1019410796 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:16.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f1054c19640 1 --2- 172.21.15.40:0/1019410796 >> v2:172.21.15.86:3300/0 conn(0x7f1050159dd0 0x7f10500c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:16.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f104ffff640 1 --2- 172.21.15.40:0/1019410796 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10500adec0 0x7f10500c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:16.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f104f7fe640 1 -- 172.21.15.40:0/1019410796 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472808020 0 0) 0x7f105015b490 con 0x7f105006b800 2023-10-16T08:39:16.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f104f7fe640 1 -- 172.21.15.40:0/1019410796 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1038005740 con 0x7f105006b800 2023-10-16T08:39:16.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f104f7fe640 1 -- 172.21.15.40:0/1019410796 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1961117825 0 0) 0x7f1038005740 con 0x7f105006b800 2023-10-16T08:39:16.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f104f7fe640 1 -- 172.21.15.40:0/1019410796 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10500adec0 msgr2=0x7f10500c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:16.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f104f7fe640 1 --2- 172.21.15.40:0/1019410796 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10500adec0 0x7f10500c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f104f7fe640 1 -- 172.21.15.40:0/1019410796 >> v2:172.21.15.86:3300/0 conn(0x7f1050159dd0 msgr2=0x7f10500c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:16.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f104f7fe640 1 --2- 172.21.15.40:0/1019410796 >> v2:172.21.15.86:3300/0 conn(0x7f1050159dd0 0x7f10500c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f104f7fe640 1 -- 172.21.15.40:0/1019410796 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105006cf50 con 0x7f105006b800 2023-10-16T08:39:16.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f104f7fe640 1 -- 172.21.15.40:0/1019410796 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f1048001a00 con 0x7f105006b800 2023-10-16T08:39:16.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f104f7fe640 1 -- 172.21.15.40:0/1019410796 >> v1:172.21.15.40:6789/0 conn(0x7f105006b800 legacy=0x7f10500cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:16.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f104f7fe640 1 --2- 172.21.15.40:0/1019410796 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10500adec0 0x7f1038005c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f104f7fe640 1 --2- 172.21.15.40:0/1019410796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1038006180 0x7f10380069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f104f7fe640 1 --2- 172.21.15.40:0/1019410796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1038006f20 0x7f10380073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f104f7fe640 1 -- 172.21.15.40:0/1019410796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105006cd70 con 0x7f1038006f20 2023-10-16T08:39:16.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f104f7fe640 1 -- 172.21.15.40:0/1019410796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105006d230 con 0x7f1038006180 2023-10-16T08:39:16.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f104f7fe640 1 -- 172.21.15.40:0/1019410796 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103800dbd0 con 0x7f10500adec0 2023-10-16T08:39:16.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f105541a640 1 --2- 172.21.15.40:0/1019410796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1038006f20 0x7f10380073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:16.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f105541a640 1 -- 172.21.15.40:0/1019410796 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10500adec0 msgr2=0x7f1038005c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:16.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f105541a640 1 --2- 172.21.15.40:0/1019410796 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10500adec0 0x7f1038005c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f105541a640 1 -- 172.21.15.40:0/1019410796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1038006180 msgr2=0x7f10380069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:16.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f105541a640 1 --2- 172.21.15.40:0/1019410796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1038006180 0x7f10380069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.211+0000 7f105541a640 1 -- 172.21.15.40:0/1019410796 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1038005740 con 0x7f1038006f20 2023-10-16T08:39:16.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.215+0000 7f105541a640 1 --2- 172.21.15.40:0/1019410796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1038006f20 0x7f10380073a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f10500ce1a0 tx=0x7f104801d380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf14008bd848eac server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:16.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.215+0000 7f104f7fe640 1 -- 172.21.15.40:0/1019410796 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10480ad630 con 0x7f1038006f20 2023-10-16T08:39:16.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.215+0000 7f104f7fe640 1 -- 172.21.15.40:0/1019410796 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10480a8070 con 0x7f1038006f20 2023-10-16T08:39:16.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.215+0000 7f104f7fe640 1 -- 172.21.15.40:0/1019410796 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10480ad9d0 con 0x7f1038006f20 2023-10-16T08:39:16.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.215+0000 7f1056e7d640 1 -- 172.21.15.40:0/1019410796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1038006f20 msgr2=0x7f10380073a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:16.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.215+0000 7f1056e7d640 1 --2- 172.21.15.40:0/1019410796 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1038006f20 0x7f10380073a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f10500ce1a0 tx=0x7f104801d380 comp rx=0 tx=0).stop 2023-10-16T08:39:16.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.215+0000 7f105541a640 1 -- 172.21.15.40:0/1019410796 reap_dead start 2023-10-16T08:39:16.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.215+0000 7f1056e7d640 1 -- 172.21.15.40:0/1019410796 shutdown_connections 2023-10-16T08:39:16.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.215+0000 7f1056e7d640 1 -- 172.21.15.40:0/1019410796 >> 172.21.15.40:0/1019410796 conn(0x7f10500ce890 msgr2=0x7f10500ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:16.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.215+0000 7f1056e7d640 1 -- 172.21.15.40:0/1019410796 shutdown_connections 2023-10-16T08:39:16.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.215+0000 7f1056e7d640 1 -- 172.21.15.40:0/1019410796 wait complete. 2023-10-16T08:39:16.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.215+0000 7f1056e7d640 1 Processor -- start 2023-10-16T08:39:16.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.215+0000 7f1056e7d640 1 -- start start 2023-10-16T08:39:16.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f1056e7d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1038006f20 0x7f10501ee260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f1056e7d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1050142690 0x7f10500a85b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f1056e7d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10500a8af0 0x7f10500c6ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f1056e7d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1050142b20 con 0x7f1038006f20 2023-10-16T08:39:16.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f1056e7d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1050142e00 con 0x7f1050142690 2023-10-16T08:39:16.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f1056e7d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10501430e0 con 0x7f10500a8af0 2023-10-16T08:39:16.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f1054c19640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1038006f20 0x7f10501ee260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:16.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f1054c19640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1038006f20 0x7f10501ee260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44730/0 (socket says 172.21.15.40:44730) 2023-10-16T08:39:16.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f1054c19640 1 -- 172.21.15.40:0/295936353 learned_addr learned my addr 172.21.15.40:0/295936353 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:16.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f105541a640 1 --2- 172.21.15.40:0/295936353 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10500a8af0 0x7f10500c6ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:16.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f104ffff640 1 --2- 172.21.15.40:0/295936353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1050142690 0x7f10500a85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:16.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f1054c19640 1 -- 172.21.15.40:0/295936353 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10500a8af0 msgr2=0x7f10500c6ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:16.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f1054c19640 1 --2- 172.21.15.40:0/295936353 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10500a8af0 0x7f10500c6ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f1054c19640 1 -- 172.21.15.40:0/295936353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1050142690 msgr2=0x7f10500a85b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:16.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f1054c19640 1 --2- 172.21.15.40:0/295936353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1050142690 0x7f10500a85b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f1054c19640 1 -- 172.21.15.40:0/295936353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10501433c0 con 0x7f1038006f20 2023-10-16T08:39:16.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f104ffff640 1 --2- 172.21.15.40:0/295936353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1050142690 0x7f10500a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:16.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f105541a640 1 --2- 172.21.15.40:0/295936353 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10500a8af0 0x7f10500c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:16.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f1054c19640 1 --2- 172.21.15.40:0/295936353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1038006f20 0x7f10501ee260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1040009fd0 tx=0x7f104000ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:16.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f104d7fa640 1 -- 172.21.15.40:0/295936353 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104000eea0 con 0x7f1038006f20 2023-10-16T08:39:16.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f104d7fa640 1 -- 172.21.15.40:0/295936353 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104000c940 con 0x7f1038006f20 2023-10-16T08:39:16.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f104d7fa640 1 -- 172.21.15.40:0/295936353 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104000ccf0 con 0x7f1038006f20 2023-10-16T08:39:16.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f1056e7d640 1 -- 172.21.15.40:0/295936353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10501436a0 con 0x7f1038006f20 2023-10-16T08:39:16.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f1056e7d640 1 -- 172.21.15.40:0/295936353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10500c71c0 con 0x7f1038006f20 2023-10-16T08:39:16.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f1056e7d640 1 -- 172.21.15.40:0/295936353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1050067a20 con 0x7f1038006f20 2023-10-16T08:39:16.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f104d7fa640 1 -- 172.21.15.40:0/295936353 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1040013470 con 0x7f1038006f20 2023-10-16T08:39:16.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f104d7fa640 1 --2- 172.21.15.40:0/295936353 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f103c0664a0 0x7f103c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f104d7fa640 1 -- 172.21.15.40:0/295936353 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1040083f00 con 0x7f1038006f20 2023-10-16T08:39:16.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.219+0000 7f104ffff640 1 --2- 172.21.15.40:0/295936353 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f103c0664a0 0x7f103c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:16.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.223+0000 7f104ffff640 1 --2- 172.21.15.40:0/295936353 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f103c0664a0 0x7f103c068960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f10440047c0 tx=0x7f1044008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:16.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.223+0000 7f104d7fa640 1 -- 172.21.15.40:0/295936353 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f1040051410 con 0x7f1038006f20 2023-10-16T08:39:16.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.383+0000 7f1056e7d640 1 -- 172.21.15.40:0/295936353 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f10500ce690 con 0x7f1038006f20 2023-10-16T08:39:16.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.383+0000 7f104d7fa640 1 -- 172.21.15.40:0/295936353 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1040054a30 con 0x7f1038006f20 2023-10-16T08:39:16.390 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-16T08:39:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.387+0000 7f1056e7d640 1 -- 172.21.15.40:0/295936353 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f103c0664a0 msgr2=0x7f103c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:16.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.387+0000 7f1056e7d640 1 --2- 172.21.15.40:0/295936353 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f103c0664a0 0x7f103c068960 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f10440047c0 tx=0x7f1044008040 comp rx=0 tx=0).stop 2023-10-16T08:39:16.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.387+0000 7f1056e7d640 1 -- 172.21.15.40:0/295936353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1038006f20 msgr2=0x7f10501ee260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:16.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.387+0000 7f1056e7d640 1 --2- 172.21.15.40:0/295936353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1038006f20 0x7f10501ee260 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1040009fd0 tx=0x7f104000ec50 comp rx=0 tx=0).stop 2023-10-16T08:39:16.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.387+0000 7f1056e7d640 1 -- 172.21.15.40:0/295936353 shutdown_connections 2023-10-16T08:39:16.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.387+0000 7f1056e7d640 1 --2- 172.21.15.40:0/295936353 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f10500a8af0 0x7f10500c6ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.387+0000 7f1056e7d640 1 --2- 172.21.15.40:0/295936353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1050142690 0x7f10500a85b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.387+0000 7f1056e7d640 1 --2- 172.21.15.40:0/295936353 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f103c0664a0 0x7f103c068960 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.387+0000 7f1056e7d640 1 --2- 172.21.15.40:0/295936353 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1038006f20 0x7f10501ee260 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.387+0000 7f1056e7d640 1 -- 172.21.15.40:0/295936353 >> 172.21.15.40:0/295936353 conn(0x7f10500ce890 msgr2=0x7f1050158120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:16.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.387+0000 7f1056e7d640 1 -- 172.21.15.40:0/295936353 shutdown_connections 2023-10-16T08:39:16.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.387+0000 7f1056e7d640 1 -- 172.21.15.40:0/295936353 wait complete. 2023-10-16T08:39:16.409 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:39:16.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.527+0000 7faf4b601640 1 Processor -- start 2023-10-16T08:39:16.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.527+0000 7faf4b601640 1 -- start start 2023-10-16T08:39:16.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.527+0000 7faf4b601640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf440adad0 0x7faf440b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.527+0000 7faf4b601640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faf440cae10 0x7faf4415a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.527+0000 7faf4b601640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf440c98b0 con 0x7faf440adeb0 2023-10-16T08:39:16.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.527+0000 7faf4b601640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf440b47c0 con 0x7faf440cae10 2023-10-16T08:39:16.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.527+0000 7faf4b601640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf440c8f80 con 0x7faf440adad0 2023-10-16T08:39:16.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.527+0000 7faf49b9e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faf440cae10 0x7faf4415a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:16.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.527+0000 7faf49b9e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7faf440cae10 0x7faf4415a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44740/0 (socket says 172.21.15.40:44740) 2023-10-16T08:39:16.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.527+0000 7faf49b9e640 1 -- 172.21.15.40:0/3767673625 learned_addr learned my addr 172.21.15.40:0/3767673625 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:16.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.527+0000 7faf4939d640 1 --2- 172.21.15.40:0/3767673625 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf440adad0 0x7faf440b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:16.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.527+0000 7faf49b9e640 1 -- 172.21.15.40:0/3767673625 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf440adad0 msgr2=0x7faf440b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:16.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.527+0000 7faf49b9e640 1 --2- 172.21.15.40:0/3767673625 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf440adad0 0x7faf440b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.527+0000 7faf49b9e640 1 -- 172.21.15.40:0/3767673625 >> v1:172.21.15.86:6789/0 conn(0x7faf440adeb0 legacy=0x7faf440ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:16.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.527+0000 7faf49b9e640 1 -- 172.21.15.40:0/3767673625 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf440c9260 con 0x7faf440cae10 2023-10-16T08:39:16.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf49b9e640 1 --2- 172.21.15.40:0/3767673625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf440cae10 0x7faf4415a500 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009e50 tx=0x7faf3c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18bdfca45f5c149 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:16.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf3bfff640 1 -- 172.21.15.40:0/3767673625 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c0a58d0 con 0x7faf440cae10 2023-10-16T08:39:16.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf3bfff640 1 -- 172.21.15.40:0/3767673625 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3c0a5a30 con 0x7faf440cae10 2023-10-16T08:39:16.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf3bfff640 1 -- 172.21.15.40:0/3767673625 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c0a5dc0 con 0x7faf440cae10 2023-10-16T08:39:16.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf4b601640 1 -- 172.21.15.40:0/3767673625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf440cae10 msgr2=0x7faf4415a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:16.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf4b601640 1 --2- 172.21.15.40:0/3767673625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf440cae10 0x7faf4415a500 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7faf3c009e50 tx=0x7faf3c01c9e0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf4b601640 1 -- 172.21.15.40:0/3767673625 shutdown_connections 2023-10-16T08:39:16.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf4b601640 1 --2- 172.21.15.40:0/3767673625 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf440cae10 0x7faf4415a500 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf4b601640 1 --2- 172.21.15.40:0/3767673625 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf440adad0 0x7faf440b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf4b601640 1 -- 172.21.15.40:0/3767673625 >> 172.21.15.40:0/3767673625 conn(0x7faf440ce890 msgr2=0x7faf440ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:16.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf4b601640 1 -- 172.21.15.40:0/3767673625 shutdown_connections 2023-10-16T08:39:16.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf4b601640 1 -- 172.21.15.40:0/3767673625 wait complete. 2023-10-16T08:39:16.543 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf4b601640 1 Processor -- start 2023-10-16T08:39:16.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf4b601640 1 -- start start 2023-10-16T08:39:16.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf4b601640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf440adad0 0x7faf440c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf4b601640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf440adeb0 0x7faf4414fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.544 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf4b601640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf440cae10 0x7faf44153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf4b601640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf4414f9a0 con 0x7faf440cae10 2023-10-16T08:39:16.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf4b601640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf440c6730 con 0x7faf440adad0 2023-10-16T08:39:16.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf4b601640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf440c6990 con 0x7faf440adeb0 2023-10-16T08:39:16.545 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf49b9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf440cae10 0x7faf44153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:16.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf49b9e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf440cae10 0x7faf44153080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44752/0 (socket says 172.21.15.40:44752) 2023-10-16T08:39:16.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.531+0000 7faf49b9e640 1 -- 172.21.15.40:0/2837754706 learned_addr learned my addr 172.21.15.40:0/2837754706 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:16.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.535+0000 7faf48b9c640 1 --2- 172.21.15.40:0/2837754706 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf440adeb0 0x7faf4414fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:16.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.535+0000 7faf4939d640 1 --2- 172.21.15.40:0/2837754706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf440adad0 0x7faf440c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:16.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.535+0000 7faf49b9e640 1 -- 172.21.15.40:0/2837754706 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf440adeb0 msgr2=0x7faf4414fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:16.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.535+0000 7faf49b9e640 1 --2- 172.21.15.40:0/2837754706 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf440adeb0 0x7faf4414fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.535+0000 7faf49b9e640 1 -- 172.21.15.40:0/2837754706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf440adad0 msgr2=0x7faf440c83d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:16.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.535+0000 7faf49b9e640 1 --2- 172.21.15.40:0/2837754706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf440adad0 0x7faf440c83d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.535+0000 7faf49b9e640 1 -- 172.21.15.40:0/2837754706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf3c009b30 con 0x7faf440cae10 2023-10-16T08:39:16.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.535+0000 7faf49b9e640 1 --2- 172.21.15.40:0/2837754706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf440cae10 0x7faf44153080 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0af040 tx=0x7faf3c019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:16.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.535+0000 7faf39ffb640 1 -- 172.21.15.40:0/2837754706 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c0b8670 con 0x7faf440cae10 2023-10-16T08:39:16.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.535+0000 7faf39ffb640 1 -- 172.21.15.40:0/2837754706 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf3c0b87d0 con 0x7faf440cae10 2023-10-16T08:39:16.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.535+0000 7faf4b601640 1 -- 172.21.15.40:0/2837754706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf4415b250 con 0x7faf440cae10 2023-10-16T08:39:16.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.535+0000 7faf39ffb640 1 -- 172.21.15.40:0/2837754706 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf3c0b8b60 con 0x7faf440cae10 2023-10-16T08:39:16.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.535+0000 7faf4b601640 1 -- 172.21.15.40:0/2837754706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf4414ed90 con 0x7faf440cae10 2023-10-16T08:39:16.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.535+0000 7faf4b601640 1 -- 172.21.15.40:0/2837754706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf44067a20 con 0x7faf440cae10 2023-10-16T08:39:16.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.535+0000 7faf39ffb640 1 -- 172.21.15.40:0/2837754706 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7faf3c0b8cc0 con 0x7faf440cae10 2023-10-16T08:39:16.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.535+0000 7faf39ffb640 1 --2- 172.21.15.40:0/2837754706 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7faf280664d0 0x7faf28068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.535+0000 7faf39ffb640 1 -- 172.21.15.40:0/2837754706 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faf3c11f540 con 0x7faf440cae10 2023-10-16T08:39:16.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.535+0000 7faf4939d640 1 --2- 172.21.15.40:0/2837754706 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7faf280664d0 0x7faf28068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:16.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.535+0000 7faf4939d640 1 --2- 172.21.15.40:0/2837754706 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7faf280664d0 0x7faf28068990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faf3400a970 tx=0x7faf34008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:16.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.539+0000 7faf39ffb640 1 -- 172.21.15.40:0/2837754706 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7faf3c0ecad0 con 0x7faf440cae10 2023-10-16T08:39:16.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.695+0000 7faf4b601640 1 -- 172.21.15.40:0/2837754706 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faf4404e9c0 con 0x7faf440cae10 2023-10-16T08:39:16.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.699+0000 7faf39ffb640 1 -- 172.21.15.40:0/2837754706 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7faf3c0f00f0 con 0x7faf440cae10 2023-10-16T08:39:16.703 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-16T08:39:16.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.699+0000 7faf4b601640 1 -- 172.21.15.40:0/2837754706 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7faf280664d0 msgr2=0x7faf28068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:16.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.699+0000 7faf4b601640 1 --2- 172.21.15.40:0/2837754706 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7faf280664d0 0x7faf28068990 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7faf3400a970 tx=0x7faf34008040 comp rx=0 tx=0).stop 2023-10-16T08:39:16.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.699+0000 7faf4b601640 1 -- 172.21.15.40:0/2837754706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf440cae10 msgr2=0x7faf44153080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:16.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.699+0000 7faf4b601640 1 --2- 172.21.15.40:0/2837754706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf440cae10 0x7faf44153080 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7faf3c0af040 tx=0x7faf3c019780 comp rx=0 tx=0).stop 2023-10-16T08:39:16.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.699+0000 7faf4b601640 1 -- 172.21.15.40:0/2837754706 shutdown_connections 2023-10-16T08:39:16.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.699+0000 7faf4b601640 1 --2- 172.21.15.40:0/2837754706 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7faf280664d0 0x7faf28068990 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.699+0000 7faf4b601640 1 --2- 172.21.15.40:0/2837754706 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7faf440cae10 0x7faf44153080 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.699+0000 7faf4b601640 1 --2- 172.21.15.40:0/2837754706 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7faf440adeb0 0x7faf4414fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.699+0000 7faf4b601640 1 --2- 172.21.15.40:0/2837754706 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf440adad0 0x7faf440c83d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.699+0000 7faf4b601640 1 -- 172.21.15.40:0/2837754706 >> 172.21.15.40:0/2837754706 conn(0x7faf440ce890 msgr2=0x7faf44154480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:16.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.699+0000 7faf4b601640 1 -- 172.21.15.40:0/2837754706 shutdown_connections 2023-10-16T08:39:16.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.699+0000 7faf4b601640 1 -- 172.21.15.40:0/2837754706 wait complete. 2023-10-16T08:39:16.721 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:39:16.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.843+0000 7fb2b1f74640 1 Processor -- start 2023-10-16T08:39:16.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.843+0000 7fb2b1f74640 1 -- start start 2023-10-16T08:39:16.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.843+0000 7fb2b1f74640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2ac15cc90 0x7fb2ac0b86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.843+0000 7fb2b1f74640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2ac0cd810 0x7fb2ac0cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.843+0000 7fb2b1f74640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2ac0d16b0 con 0x7fb2ac0d2750 2023-10-16T08:39:16.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.843+0000 7fb2b1f74640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2ac0ce1e0 con 0x7fb2ac0cd810 2023-10-16T08:39:16.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.843+0000 7fb2b1f74640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac0b6c80 con 0x7fb2ac15cc90 2023-10-16T08:39:16.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.843+0000 7fb2abfff640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2ac0cd810 0x7fb2ac0cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:16.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.843+0000 7fb2ab7fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2ac15cc90 0x7fb2ac0b86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:16.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.843+0000 7fb2abfff640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2ac0cd810 0x7fb2ac0cdca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.40:40232/0 (socket says 172.21.15.40:40232) 2023-10-16T08:39:16.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.843+0000 7fb2abfff640 1 -- 172.21.15.40:0/3339891394 learned_addr learned my addr 172.21.15.40:0/3339891394 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:16.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.843+0000 7fb2aaffd640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7fb2ac0d2750 0x7fb2ac0d0ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.40:34652/0 (socket says 172.21.15.40:34652) 2023-10-16T08:39:16.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.843+0000 7fb2abfff640 1 -- 172.21.15.40:0/3339891394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2ac15cc90 msgr2=0x7fb2ac0b86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:16.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.843+0000 7fb2abfff640 1 --2- 172.21.15.40:0/3339891394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2ac15cc90 0x7fb2ac0b86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.843+0000 7fb2abfff640 1 -- 172.21.15.40:0/3339891394 >> v1:172.21.15.86:6789/0 conn(0x7fb2ac0d2750 legacy=0x7fb2ac0d0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:16.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.843+0000 7fb2abfff640 1 -- 172.21.15.40:0/3339891394 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ac0d1990 con 0x7fb2ac0cd810 2023-10-16T08:39:16.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.847+0000 7fb2abfff640 1 --2- 172.21.15.40:0/3339891394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2ac0cd810 0x7fb2ac0cdca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009e50 tx=0x7fb2a401c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6f2cf369238ed1a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:16.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.847+0000 7fb2aa7fc640 1 -- 172.21.15.40:0/3339891394 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a40a58a0 con 0x7fb2ac0cd810 2023-10-16T08:39:16.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.847+0000 7fb2aa7fc640 1 -- 172.21.15.40:0/3339891394 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a40a5a00 con 0x7fb2ac0cd810 2023-10-16T08:39:16.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.847+0000 7fb2aa7fc640 1 -- 172.21.15.40:0/3339891394 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a40a5d90 con 0x7fb2ac0cd810 2023-10-16T08:39:16.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.847+0000 7fb2b1f74640 1 -- 172.21.15.40:0/3339891394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2ac0cd810 msgr2=0x7fb2ac0cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:16.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.847+0000 7fb2b1f74640 1 --2- 172.21.15.40:0/3339891394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2ac0cd810 0x7fb2ac0cdca0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fb2a4009e50 tx=0x7fb2a401c9e0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.847+0000 7fb2b1f74640 1 -- 172.21.15.40:0/3339891394 shutdown_connections 2023-10-16T08:39:16.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.847+0000 7fb2b1f74640 1 --2- 172.21.15.40:0/3339891394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2ac0cd810 0x7fb2ac0cdca0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.847+0000 7fb2b1f74640 1 --2- 172.21.15.40:0/3339891394 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2ac15cc90 0x7fb2ac0b86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.847+0000 7fb2b1f74640 1 -- 172.21.15.40:0/3339891394 >> 172.21.15.40:0/3339891394 conn(0x7fb2ac0cf5c0 msgr2=0x7fb2ac0cf9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:16.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.847+0000 7fb2b1f74640 1 -- 172.21.15.40:0/3339891394 shutdown_connections 2023-10-16T08:39:16.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.847+0000 7fb2b1f74640 1 -- 172.21.15.40:0/3339891394 wait complete. 2023-10-16T08:39:16.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.847+0000 7fb2b1f74640 1 Processor -- start 2023-10-16T08:39:16.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.847+0000 7fb2b1f74640 1 -- start start 2023-10-16T08:39:16.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2b1f74640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2ac0cd810 0x7fb2ac14f070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2b1f74640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2ac15cc90 0x7fb2ac155140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2b1f74640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2ac155680 0x7fb2ac0cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2b1f74640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac0c7a90 con 0x7fb2ac15cc90 2023-10-16T08:39:16.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2b1f74640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac0c8050 con 0x7fb2ac0cd810 2023-10-16T08:39:16.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2b1f74640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ac155b90 con 0x7fb2ac155680 2023-10-16T08:39:16.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2aaffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2ac15cc90 0x7fb2ac155140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:16.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2aaffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2ac15cc90 0x7fb2ac155140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44754/0 (socket says 172.21.15.40:44754) 2023-10-16T08:39:16.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2aaffd640 1 -- 172.21.15.40:0/1128951130 learned_addr learned my addr 172.21.15.40:0/1128951130 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:16.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2abfff640 1 --2- 172.21.15.40:0/1128951130 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2ac155680 0x7fb2ac0cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:16.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2ab7fe640 1 --2- 172.21.15.40:0/1128951130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2ac0cd810 0x7fb2ac14f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2aaffd640 1 -- 172.21.15.40:0/1128951130 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2ac155680 msgr2=0x7fb2ac0cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2aaffd640 1 --2- 172.21.15.40:0/1128951130 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2ac155680 0x7fb2ac0cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2aaffd640 1 -- 172.21.15.40:0/1128951130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2ac0cd810 msgr2=0x7fb2ac14f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2aaffd640 1 --2- 172.21.15.40:0/1128951130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2ac0cd810 0x7fb2ac14f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2aaffd640 1 -- 172.21.15.40:0/1128951130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a4009b30 con 0x7fb2ac15cc90 2023-10-16T08:39:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2abfff640 1 --2- 172.21.15.40:0/1128951130 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2ac155680 0x7fb2ac0cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:16.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2aaffd640 1 --2- 172.21.15.40:0/1128951130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2ac15cc90 0x7fb2ac155140 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000c460 tx=0x7fb2a000c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:16.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb293fff640 1 -- 172.21.15.40:0/1128951130 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0007480 con 0x7fb2ac15cc90 2023-10-16T08:39:16.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb293fff640 1 -- 172.21.15.40:0/1128951130 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a00075e0 con 0x7fb2ac15cc90 2023-10-16T08:39:16.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2b1f74640 1 -- 172.21.15.40:0/1128951130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2ac155e70 con 0x7fb2ac15cc90 2023-10-16T08:39:16.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb293fff640 1 -- 172.21.15.40:0/1128951130 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a0004a30 con 0x7fb2ac15cc90 2023-10-16T08:39:16.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2b1f74640 1 -- 172.21.15.40:0/1128951130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2ac14faf0 con 0x7fb2ac15cc90 2023-10-16T08:39:16.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2b1f74640 1 -- 172.21.15.40:0/1128951130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ac0c81e0 con 0x7fb2ac15cc90 2023-10-16T08:39:16.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb293fff640 1 -- 172.21.15.40:0/1128951130 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb2a0004b90 con 0x7fb2ac15cc90 2023-10-16T08:39:16.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb293fff640 1 --2- 172.21.15.40:0/1128951130 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb2980664d0 0x7fb298068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:16.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb293fff640 1 -- 172.21.15.40:0/1128951130 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb2a00830e0 con 0x7fb2ac15cc90 2023-10-16T08:39:16.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.851+0000 7fb2ab7fe640 1 --2- 172.21.15.40:0/1128951130 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb2980664d0 0x7fb298068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:16.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.855+0000 7fb2ab7fe640 1 --2- 172.21.15.40:0/1128951130 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb2980664d0 0x7fb298068990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac150360 tx=0x7fb29c009190 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:16.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:16.855+0000 7fb293fff640 1 -- 172.21.15.40:0/1128951130 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fb2a00506a0 con 0x7fb2ac15cc90 2023-10-16T08:39:17.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.015+0000 7fb2b1f74640 1 -- 172.21.15.40:0/1128951130 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb2ac04e9c0 con 0x7fb2ac15cc90 2023-10-16T08:39:17.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.015+0000 7fb293fff640 1 -- 172.21.15.40:0/1128951130 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb2a0053cc0 con 0x7fb2ac15cc90 2023-10-16T08:39:17.021 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-16T08:39:17.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.015+0000 7fb2b1f74640 1 -- 172.21.15.40:0/1128951130 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb2980664d0 msgr2=0x7fb298068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:17.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.015+0000 7fb2b1f74640 1 --2- 172.21.15.40:0/1128951130 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb2980664d0 0x7fb298068990 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac150360 tx=0x7fb29c009190 comp rx=0 tx=0).stop 2023-10-16T08:39:17.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.015+0000 7fb2b1f74640 1 -- 172.21.15.40:0/1128951130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2ac15cc90 msgr2=0x7fb2ac155140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:17.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.015+0000 7fb2b1f74640 1 --2- 172.21.15.40:0/1128951130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2ac15cc90 0x7fb2ac155140 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb2a000c460 tx=0x7fb2a000c920 comp rx=0 tx=0).stop 2023-10-16T08:39:17.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.015+0000 7fb2b1f74640 1 -- 172.21.15.40:0/1128951130 shutdown_connections 2023-10-16T08:39:17.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.015+0000 7fb2b1f74640 1 --2- 172.21.15.40:0/1128951130 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2ac155680 0x7fb2ac0cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.015+0000 7fb2b1f74640 1 --2- 172.21.15.40:0/1128951130 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb2980664d0 0x7fb298068990 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.015+0000 7fb2b1f74640 1 --2- 172.21.15.40:0/1128951130 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2ac15cc90 0x7fb2ac155140 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.015+0000 7fb2b1f74640 1 --2- 172.21.15.40:0/1128951130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2ac0cd810 0x7fb2ac14f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.015+0000 7fb2b1f74640 1 -- 172.21.15.40:0/1128951130 >> 172.21.15.40:0/1128951130 conn(0x7fb2ac0cf5c0 msgr2=0x7fb2ac157400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:17.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.015+0000 7fb2b1f74640 1 -- 172.21.15.40:0/1128951130 shutdown_connections 2023-10-16T08:39:17.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.015+0000 7fb2b1f74640 1 -- 172.21.15.40:0/1128951130 wait complete. 2023-10-16T08:39:17.040 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:39:17.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb4122a640 1 Processor -- start 2023-10-16T08:39:17.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb4122a640 1 -- start start 2023-10-16T08:39:17.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb4122a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb3c0cddc0 0x7feb3c0d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:17.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb4122a640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb3c0b7f80 con 0x7feb3c0cd960 2023-10-16T08:39:17.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb4122a640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb3c0b7370 con 0x7feb3c15cce0 2023-10-16T08:39:17.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb4122a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c0b7650 con 0x7feb3c0cddc0 2023-10-16T08:39:17.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb3a59c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7feb3c0cd960 0x7feb3c0d0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60888/0 (socket says 172.21.15.40:60888) 2023-10-16T08:39:17.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb3a59c640 1 -- 172.21.15.40:0/2300578458 learned_addr learned my addr 172.21.15.40:0/2300578458 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:17.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb3b59e640 1 --2- 172.21.15.40:0/2300578458 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb3c0cddc0 0x7feb3c0d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:17.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb39d9b640 1 -- 172.21.15.40:0/2300578458 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3003489247 0 0) 0x7feb3c0b7f80 con 0x7feb3c0cd960 2023-10-16T08:39:17.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb39d9b640 1 -- 172.21.15.40:0/2300578458 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb24003530 con 0x7feb3c0cd960 2023-10-16T08:39:17.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb39d9b640 1 -- 172.21.15.40:0/2300578458 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2610700158 0 0) 0x7feb24003530 con 0x7feb3c0cd960 2023-10-16T08:39:17.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb39d9b640 1 -- 172.21.15.40:0/2300578458 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb3c0cddc0 msgr2=0x7feb3c0d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:17.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb39d9b640 1 --2- 172.21.15.40:0/2300578458 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb3c0cddc0 0x7feb3c0d1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb39d9b640 1 -- 172.21.15.40:0/2300578458 >> v1:172.21.15.86:6789/0 conn(0x7feb3c15cce0 legacy=0x7feb3c0b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:17.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb39d9b640 1 -- 172.21.15.40:0/2300578458 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c0b7930 con 0x7feb3c0cd960 2023-10-16T08:39:17.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb39d9b640 1 -- 172.21.15.40:0/2300578458 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7feb30001680 con 0x7feb3c0cd960 2023-10-16T08:39:17.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb39d9b640 1 -- 172.21.15.40:0/2300578458 >> v1:172.21.15.40:6789/0 conn(0x7feb3c0cd960 legacy=0x7feb3c0d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:17.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb39d9b640 1 --2- 172.21.15.40:0/2300578458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb24003a20 0x7feb24003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:17.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb39d9b640 1 --2- 172.21.15.40:0/2300578458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb240047d0 0x7feb24004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:17.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb39d9b640 1 --2- 172.21.15.40:0/2300578458 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb3c0cddc0 0x7feb24005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:17.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb39d9b640 1 -- 172.21.15.40:0/2300578458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c0b7650 con 0x7feb240047d0 2023-10-16T08:39:17.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb39d9b640 1 -- 172.21.15.40:0/2300578458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb2400ba90 con 0x7feb24003a20 2023-10-16T08:39:17.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb39d9b640 1 -- 172.21.15.40:0/2300578458 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2400bc50 con 0x7feb3c0cddc0 2023-10-16T08:39:17.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb3a59c640 1 --2- 172.21.15.40:0/2300578458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb240047d0 0x7feb24004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:17.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb3ad9d640 1 --2- 172.21.15.40:0/2300578458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb24003a20 0x7feb24003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:17.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb3a59c640 1 -- 172.21.15.40:0/2300578458 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb3c0cddc0 msgr2=0x7feb24005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:17.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb3a59c640 1 --2- 172.21.15.40:0/2300578458 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb3c0cddc0 0x7feb24005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb3a59c640 1 -- 172.21.15.40:0/2300578458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb24003a20 msgr2=0x7feb24003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:17.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb3a59c640 1 --2- 172.21.15.40:0/2300578458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb24003a20 0x7feb24003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb3a59c640 1 -- 172.21.15.40:0/2300578458 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb24003530 con 0x7feb240047d0 2023-10-16T08:39:17.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.159+0000 7feb3ad9d640 1 --2- 172.21.15.40:0/2300578458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb24003a20 0x7feb24003e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:17.178 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.163+0000 7feb3a59c640 1 --2- 172.21.15.40:0/2300578458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb240047d0 0x7feb24004c50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feb3000c240 tx=0x7feb3001f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d09001b37a82becf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:17.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.163+0000 7feb39d9b640 1 -- 172.21.15.40:0/2300578458 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb300058e0 con 0x7feb240047d0 2023-10-16T08:39:17.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.163+0000 7feb39d9b640 1 -- 172.21.15.40:0/2300578458 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb30005a40 con 0x7feb240047d0 2023-10-16T08:39:17.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.163+0000 7feb39d9b640 1 -- 172.21.15.40:0/2300578458 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb30005e30 con 0x7feb240047d0 2023-10-16T08:39:17.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.163+0000 7feb4122a640 1 -- 172.21.15.40:0/2300578458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb240047d0 msgr2=0x7feb24004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:17.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.163+0000 7feb4122a640 1 --2- 172.21.15.40:0/2300578458 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb240047d0 0x7feb24004c50 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7feb3000c240 tx=0x7feb3001f3b0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.163+0000 7feb3b59e640 1 -- 172.21.15.40:0/2300578458 reap_dead start 2023-10-16T08:39:17.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.163+0000 7feb4122a640 1 -- 172.21.15.40:0/2300578458 shutdown_connections 2023-10-16T08:39:17.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.163+0000 7feb4122a640 1 -- 172.21.15.40:0/2300578458 >> 172.21.15.40:0/2300578458 conn(0x7feb3c06a290 msgr2=0x7feb3c06a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:17.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.163+0000 7feb4122a640 1 -- 172.21.15.40:0/2300578458 shutdown_connections 2023-10-16T08:39:17.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.163+0000 7feb4122a640 1 -- 172.21.15.40:0/2300578458 wait complete. 2023-10-16T08:39:17.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.163+0000 7feb4122a640 1 Processor -- start 2023-10-16T08:39:17.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.163+0000 7feb4122a640 1 -- start start 2023-10-16T08:39:17.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb4122a640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb240047d0 0x7feb3c150380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:17.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb4122a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb3c0c64d0 0x7feb3c0c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:17.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb4122a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb3c15d0c0 0x7feb3c15d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:17.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb4122a640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c0c7cc0 con 0x7feb3c0c64d0 2023-10-16T08:39:17.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb4122a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c15dab0 con 0x7feb3c15d0c0 2023-10-16T08:39:17.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb4122a640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb3c0c7040 con 0x7feb240047d0 2023-10-16T08:39:17.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb3a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb3c0c64d0 0x7feb3c0c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:17.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb3a59c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb3c0c64d0 0x7feb3c0c6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44768/0 (socket says 172.21.15.40:44768) 2023-10-16T08:39:17.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb3a59c640 1 -- 172.21.15.40:0/1557412589 learned_addr learned my addr 172.21.15.40:0/1557412589 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:17.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb3ad9d640 1 --2- 172.21.15.40:0/1557412589 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb240047d0 0x7feb3c150380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:17.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb3b59e640 1 --2- 172.21.15.40:0/1557412589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb3c15d0c0 0x7feb3c15d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:17.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb3a59c640 1 -- 172.21.15.40:0/1557412589 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb240047d0 msgr2=0x7feb3c150380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:17.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb3a59c640 1 --2- 172.21.15.40:0/1557412589 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb240047d0 0x7feb3c150380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb3a59c640 1 -- 172.21.15.40:0/1557412589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb3c15d0c0 msgr2=0x7feb3c15d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:17.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb3a59c640 1 --2- 172.21.15.40:0/1557412589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb3c15d0c0 0x7feb3c15d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb3a59c640 1 -- 172.21.15.40:0/1557412589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb3c0c7320 con 0x7feb3c0c64d0 2023-10-16T08:39:17.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb3a59c640 1 --2- 172.21.15.40:0/1557412589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb3c0c64d0 0x7feb3c0c6950 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feb3000c240 tx=0x7feb24005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb237fe640 1 -- 172.21.15.40:0/1557412589 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb300b7070 con 0x7feb3c0c64d0 2023-10-16T08:39:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb237fe640 1 -- 172.21.15.40:0/1557412589 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb30001440 con 0x7feb3c0c64d0 2023-10-16T08:39:17.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb237fe640 1 -- 172.21.15.40:0/1557412589 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb300047f0 con 0x7feb3c0c64d0 2023-10-16T08:39:17.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb4122a640 1 -- 172.21.15.40:0/1557412589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb3c0c7600 con 0x7feb3c0c64d0 2023-10-16T08:39:17.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb4122a640 1 -- 172.21.15.40:0/1557412589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb3c15e9b0 con 0x7feb3c0c64d0 2023-10-16T08:39:17.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb4122a640 1 -- 172.21.15.40:0/1557412589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb3c0c7f30 con 0x7feb3c0c64d0 2023-10-16T08:39:17.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb237fe640 1 -- 172.21.15.40:0/1557412589 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7feb30004970 con 0x7feb3c0c64d0 2023-10-16T08:39:17.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb237fe640 1 --2- 172.21.15.40:0/1557412589 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7feb280664d0 0x7feb28068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:17.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb237fe640 1 -- 172.21.15.40:0/1557412589 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb30120de0 con 0x7feb3c0c64d0 2023-10-16T08:39:17.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.167+0000 7feb3ad9d640 1 --2- 172.21.15.40:0/1557412589 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7feb280664d0 0x7feb28068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:17.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.171+0000 7feb3ad9d640 1 --2- 172.21.15.40:0/1557412589 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7feb280664d0 0x7feb28068990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7feb2c000f70 tx=0x7feb2c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:17.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.175+0000 7feb237fe640 1 -- 172.21.15.40:0/1557412589 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7feb300ee2f0 con 0x7feb3c0c64d0 2023-10-16T08:39:17.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.343+0000 7feb4122a640 1 -- 172.21.15.40:0/1557412589 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7feb3c04e9c0 con 0x7feb3c0c64d0 2023-10-16T08:39:17.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.343+0000 7feb237fe640 1 -- 172.21.15.40:0/1557412589 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7feb300f1910 con 0x7feb3c0c64d0 2023-10-16T08:39:17.351 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-16T08:39:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.347+0000 7feb4122a640 1 -- 172.21.15.40:0/1557412589 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7feb280664d0 msgr2=0x7feb28068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.347+0000 7feb4122a640 1 --2- 172.21.15.40:0/1557412589 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7feb280664d0 0x7feb28068990 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7feb2c000f70 tx=0x7feb2c009040 comp rx=0 tx=0).stop 2023-10-16T08:39:17.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.347+0000 7feb4122a640 1 -- 172.21.15.40:0/1557412589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb3c0c64d0 msgr2=0x7feb3c0c6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.347+0000 7feb4122a640 1 --2- 172.21.15.40:0/1557412589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb3c0c64d0 0x7feb3c0c6950 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7feb3000c240 tx=0x7feb24005800 comp rx=0 tx=0).stop 2023-10-16T08:39:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.347+0000 7feb4122a640 1 -- 172.21.15.40:0/1557412589 shutdown_connections 2023-10-16T08:39:17.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.347+0000 7feb4122a640 1 --2- 172.21.15.40:0/1557412589 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb3c15d0c0 0x7feb3c15d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.347+0000 7feb4122a640 1 --2- 172.21.15.40:0/1557412589 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7feb280664d0 0x7feb28068990 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.347+0000 7feb4122a640 1 --2- 172.21.15.40:0/1557412589 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7feb3c0c64d0 0x7feb3c0c6950 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.347+0000 7feb4122a640 1 --2- 172.21.15.40:0/1557412589 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7feb240047d0 0x7feb3c150380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.355 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.347+0000 7feb4122a640 1 -- 172.21.15.40:0/1557412589 >> 172.21.15.40:0/1557412589 conn(0x7feb3c06a290 msgr2=0x7feb3c0d2c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:17.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.347+0000 7feb4122a640 1 -- 172.21.15.40:0/1557412589 shutdown_connections 2023-10-16T08:39:17.356 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.347+0000 7feb4122a640 1 -- 172.21.15.40:0/1557412589 wait complete. 2023-10-16T08:39:17.370 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:39:17.370 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:39:17.376 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:39:17.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.539+0000 7f074110e640 1 Processor -- start 2023-10-16T08:39:17.547 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.539+0000 7f074110e640 1 -- start start 2023-10-16T08:39:17.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.543+0000 7f074110e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f073c06e100 0x7f073c0ca7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:17.548 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.543+0000 7f074110e640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f073c06d210 0x7f073c06d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:17.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.543+0000 7f074110e640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f073c070260 con 0x7f073c159bf0 2023-10-16T08:39:17.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.543+0000 7f074110e640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f073c06e600 con 0x7f073c06e100 2023-10-16T08:39:17.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.543+0000 7f074110e640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f073c06db80 con 0x7f073c06d210 2023-10-16T08:39:17.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.543+0000 7f0733fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f073c06e100 0x7f073c0ca7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:17.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.543+0000 7f0733fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f073c06e100 0x7f073c0ca7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44774/0 (socket says 172.21.15.40:44774) 2023-10-16T08:39:17.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.543+0000 7f0733fff640 1 -- 172.21.15.40:0/189430567 learned_addr learned my addr 172.21.15.40:0/189430567 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:17.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.543+0000 7f073b59e640 1 --2- 172.21.15.40:0/189430567 >> v2:172.21.15.86:3300/0 conn(0x7f073c06d210 0x7f073c06d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:17.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.543+0000 7f0733fff640 1 -- 172.21.15.40:0/189430567 >> v2:172.21.15.86:3300/0 conn(0x7f073c06d210 msgr2=0x7f073c06d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:17.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.543+0000 7f0733fff640 1 --2- 172.21.15.40:0/189430567 >> v2:172.21.15.86:3300/0 conn(0x7f073c06d210 0x7f073c06d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.543+0000 7f0733fff640 1 -- 172.21.15.40:0/189430567 >> v1:172.21.15.86:6789/0 conn(0x7f073c159bf0 legacy=0x7f073c071250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:17.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.543+0000 7f0733fff640 1 -- 172.21.15.40:0/189430567 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f073c070490 con 0x7f073c06e100 2023-10-16T08:39:17.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.543+0000 7f0733fff640 1 --2- 172.21.15.40:0/189430567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f073c06e100 0x7f073c0ca7b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f073c073000 tx=0x7f072401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26af50dbe50e5ced server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:17.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f073a59c640 1 -- 172.21.15.40:0/189430567 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07240ab5f0 con 0x7f073c06e100 2023-10-16T08:39:17.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f073a59c640 1 -- 172.21.15.40:0/189430567 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07240ab750 con 0x7f073c06e100 2023-10-16T08:39:17.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f073a59c640 1 -- 172.21.15.40:0/189430567 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07240abae0 con 0x7f073c06e100 2023-10-16T08:39:17.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f074110e640 1 -- 172.21.15.40:0/189430567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f073c06e100 msgr2=0x7f073c0ca7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:17.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f074110e640 1 --2- 172.21.15.40:0/189430567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f073c06e100 0x7f073c0ca7b0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f073c073000 tx=0x7f072401c500 comp rx=0 tx=0).stop 2023-10-16T08:39:17.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f074110e640 1 -- 172.21.15.40:0/189430567 shutdown_connections 2023-10-16T08:39:17.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f074110e640 1 --2- 172.21.15.40:0/189430567 >> v2:172.21.15.86:3300/0 conn(0x7f073c06d210 0x7f073c06d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f074110e640 1 --2- 172.21.15.40:0/189430567 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f073c06e100 0x7f073c0ca7b0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f074110e640 1 -- 172.21.15.40:0/189430567 >> 172.21.15.40:0/189430567 conn(0x7f073c0b0be0 msgr2=0x7f073c0b0ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f074110e640 1 -- 172.21.15.40:0/189430567 shutdown_connections 2023-10-16T08:39:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f074110e640 1 -- 172.21.15.40:0/189430567 wait complete. 2023-10-16T08:39:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f074110e640 1 Processor -- start 2023-10-16T08:39:17.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f074110e640 1 -- start start 2023-10-16T08:39:17.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f074110e640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f073c06d210 0x7f073c0c4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:17.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f074110e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f073c06e100 0x7f073c150c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:17.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f074110e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c159bf0 0x7f073c0c6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:17.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f074110e640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c1511b0 con 0x7f073c06e100 2023-10-16T08:39:17.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f074110e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f073c1f9550 con 0x7f073c159bf0 2023-10-16T08:39:17.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f074110e640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f073c1f9830 con 0x7f073c06d210 2023-10-16T08:39:17.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f0733fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f073c06e100 0x7f073c150c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:17.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f0733fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f073c06e100 0x7f073c150c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44778/0 (socket says 172.21.15.40:44778) 2023-10-16T08:39:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f0733fff640 1 -- 172.21.15.40:0/119334836 learned_addr learned my addr 172.21.15.40:0/119334836 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f073b59e640 1 --2- 172.21.15.40:0/119334836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c159bf0 0x7f073c0c6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:17.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f073ad9d640 1 --2- 172.21.15.40:0/119334836 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f073c06d210 0x7f073c0c4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:17.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f0733fff640 1 -- 172.21.15.40:0/119334836 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f073c06d210 msgr2=0x7f073c0c4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:17.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f0733fff640 1 --2- 172.21.15.40:0/119334836 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f073c06d210 0x7f073c0c4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f0733fff640 1 -- 172.21.15.40:0/119334836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c159bf0 msgr2=0x7f073c0c6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:17.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f0733fff640 1 --2- 172.21.15.40:0/119334836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c159bf0 0x7f073c0c6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f0733fff640 1 -- 172.21.15.40:0/119334836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07240094d0 con 0x7f073c06e100 2023-10-16T08:39:17.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f0733fff640 1 --2- 172.21.15.40:0/119334836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f073c06e100 0x7f073c150c70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0724019110 tx=0x7f072401ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:17.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.547+0000 7f07337fe640 1 -- 172.21.15.40:0/119334836 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07240be5f0 con 0x7f073c06e100 2023-10-16T08:39:17.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.551+0000 7f07337fe640 1 -- 172.21.15.40:0/119334836 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07240be750 con 0x7f073c06e100 2023-10-16T08:39:17.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.551+0000 7f074110e640 1 -- 172.21.15.40:0/119334836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f073c1f9b10 con 0x7f073c06e100 2023-10-16T08:39:17.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.551+0000 7f07337fe640 1 -- 172.21.15.40:0/119334836 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07240beb00 con 0x7f073c06e100 2023-10-16T08:39:17.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.551+0000 7f074110e640 1 -- 172.21.15.40:0/119334836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f073c0c67c0 con 0x7f073c06e100 2023-10-16T08:39:17.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.551+0000 7f074110e640 1 -- 172.21.15.40:0/119334836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f073c1f5cd0 con 0x7f073c06e100 2023-10-16T08:39:17.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.551+0000 7f07337fe640 1 -- 172.21.15.40:0/119334836 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f07240b43f0 con 0x7f073c06e100 2023-10-16T08:39:17.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.551+0000 7f07337fe640 1 --2- 172.21.15.40:0/119334836 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f071c0664a0 0x7f071c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:17.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.551+0000 7f07337fe640 1 -- 172.21.15.40:0/119334836 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f07240a8050 con 0x7f073c06e100 2023-10-16T08:39:17.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.551+0000 7f073ad9d640 1 --2- 172.21.15.40:0/119334836 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f071c0664a0 0x7f071c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:17.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.551+0000 7f073ad9d640 1 --2- 172.21.15.40:0/119334836 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f071c0664a0 0x7f071c068960 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f072c000f70 tx=0x7f072c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:17.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.555+0000 7f07337fe640 1 -- 172.21.15.40:0/119334836 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f073c1f5cd0 con 0x7f073c06e100 2023-10-16T08:39:17.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.719+0000 7f074110e640 1 -- 172.21.15.40:0/119334836 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f073c1f5cd0 con 0x7f073c06e100 2023-10-16T08:39:17.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.723+0000 7f07337fe640 1 -- 172.21.15.40:0/119334836 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f073c1f5cd0 con 0x7f073c06e100 2023-10-16T08:39:17.727 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:39:17.727 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:46.611098+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:39:17.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.723+0000 7f074110e640 1 -- 172.21.15.40:0/119334836 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f071c0664a0 msgr2=0x7f071c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.723+0000 7f074110e640 1 --2- 172.21.15.40:0/119334836 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f071c0664a0 0x7f071c068960 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f072c000f70 tx=0x7f072c009040 comp rx=0 tx=0).stop 2023-10-16T08:39:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.723+0000 7f074110e640 1 -- 172.21.15.40:0/119334836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f073c06e100 msgr2=0x7f073c150c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:17.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.723+0000 7f074110e640 1 --2- 172.21.15.40:0/119334836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f073c06e100 0x7f073c150c70 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f0724019110 tx=0x7f072401ad40 comp rx=0 tx=0).stop 2023-10-16T08:39:17.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.723+0000 7f074110e640 1 -- 172.21.15.40:0/119334836 shutdown_connections 2023-10-16T08:39:17.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.723+0000 7f074110e640 1 --2- 172.21.15.40:0/119334836 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c159bf0 0x7f073c0c6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.723+0000 7f074110e640 1 --2- 172.21.15.40:0/119334836 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f071c0664a0 0x7f071c068960 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.723+0000 7f074110e640 1 --2- 172.21.15.40:0/119334836 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f073c06e100 0x7f073c150c70 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.723+0000 7f074110e640 1 --2- 172.21.15.40:0/119334836 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f073c06d210 0x7f073c0c4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.723+0000 7f074110e640 1 -- 172.21.15.40:0/119334836 >> 172.21.15.40:0/119334836 conn(0x7f073c0b0be0 msgr2=0x7f073c0c8350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:17.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.723+0000 7f074110e640 1 -- 172.21.15.40:0/119334836 shutdown_connections 2023-10-16T08:39:17.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.723+0000 7f074110e640 1 -- 172.21.15.40:0/119334836 wait complete. 2023-10-16T08:39:17.747 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:38:33.014668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:38:43.055215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:38:43.971666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:39:17.748 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:39:17.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.867+0000 7fe83fb27640 1 Processor -- start 2023-10-16T08:39:17.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.867+0000 7fe83fb27640 1 -- start start 2023-10-16T08:39:17.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.867+0000 7fe83fb27640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe8380adad0 0x7fe8380b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:17.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.867+0000 7fe83fb27640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe8380cae10 0x7fe83815a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:17.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.867+0000 7fe83fb27640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8380c98b0 con 0x7fe8380adeb0 2023-10-16T08:39:17.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.867+0000 7fe83fb27640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8380b47c0 con 0x7fe8380adad0 2023-10-16T08:39:17.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.867+0000 7fe83fb27640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8380c8f80 con 0x7fe8380cae10 2023-10-16T08:39:17.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.867+0000 7fe83d8c3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fe8380adad0 0x7fe8380b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:17.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.867+0000 7fe83d0c2640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe8380adeb0 0x7fe8380ca700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60896/0 (socket says 172.21.15.40:60896) 2023-10-16T08:39:17.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.867+0000 7fe83d0c2640 1 -- 172.21.15.40:0/1612417477 learned_addr learned my addr 172.21.15.40:0/1612417477 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:17.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.867+0000 7fe83e0c4640 1 --2- 172.21.15.40:0/1612417477 >> v2:172.21.15.86:3300/0 conn(0x7fe8380cae10 0x7fe83815a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:17.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.867+0000 7fe83d8c3640 1 -- 172.21.15.40:0/1612417477 >> v2:172.21.15.86:3300/0 conn(0x7fe8380cae10 msgr2=0x7fe83815a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:17.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.867+0000 7fe83d8c3640 1 --2- 172.21.15.40:0/1612417477 >> v2:172.21.15.86:3300/0 conn(0x7fe8380cae10 0x7fe83815a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.867+0000 7fe83d8c3640 1 -- 172.21.15.40:0/1612417477 >> v1:172.21.15.40:6789/0 conn(0x7fe8380adeb0 legacy=0x7fe8380ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:17.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.867+0000 7fe83d8c3640 1 -- 172.21.15.40:0/1612417477 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8380c9260 con 0x7fe8380adad0 2023-10-16T08:39:17.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.867+0000 7fe83c8c1640 1 -- 172.21.15.40:0/1612417477 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3926242371 0 0) 0x7fe8380c98b0 con 0x7fe8380adeb0 2023-10-16T08:39:17.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.867+0000 7fe83e0c4640 1 --2- 172.21.15.40:0/1612417477 >> v2:172.21.15.86:3300/0 conn(0x7fe8380cae10 0x7fe83815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:17.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83d8c3640 1 --2- 172.21.15.40:0/1612417477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8380adad0 0x7fe8380b41e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe83806d3d0 tx=0x7fe83401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d5afe128c5290da server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:17.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83c8c1640 1 -- 172.21.15.40:0/1612417477 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8340ab650 con 0x7fe8380adad0 2023-10-16T08:39:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83c8c1640 1 -- 172.21.15.40:0/1612417477 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8340ab7b0 con 0x7fe8380adad0 2023-10-16T08:39:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83c8c1640 1 -- 172.21.15.40:0/1612417477 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8340abb40 con 0x7fe8380adad0 2023-10-16T08:39:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83fb27640 1 -- 172.21.15.40:0/1612417477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8380adad0 msgr2=0x7fe8380b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:17.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83fb27640 1 --2- 172.21.15.40:0/1612417477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8380adad0 0x7fe8380b41e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fe83806d3d0 tx=0x7fe83401c430 comp rx=0 tx=0).stop 2023-10-16T08:39:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83fb27640 1 -- 172.21.15.40:0/1612417477 shutdown_connections 2023-10-16T08:39:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83fb27640 1 --2- 172.21.15.40:0/1612417477 >> v2:172.21.15.86:3300/0 conn(0x7fe8380cae10 0x7fe83815a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83fb27640 1 --2- 172.21.15.40:0/1612417477 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8380adad0 0x7fe8380b41e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83fb27640 1 -- 172.21.15.40:0/1612417477 >> 172.21.15.40:0/1612417477 conn(0x7fe8380ce890 msgr2=0x7fe8380ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83fb27640 1 -- 172.21.15.40:0/1612417477 shutdown_connections 2023-10-16T08:39:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83fb27640 1 -- 172.21.15.40:0/1612417477 wait complete. 2023-10-16T08:39:17.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83fb27640 1 Processor -- start 2023-10-16T08:39:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83fb27640 1 -- start start 2023-10-16T08:39:17.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83fb27640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8380adad0 0x7fe838153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:17.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83fb27640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8380adeb0 0x7fe8380c3920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:17.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83fb27640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8380cae10 0x7fe8380c3e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83fb27640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe838153450 con 0x7fe8380cae10 2023-10-16T08:39:17.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83fb27640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8380a8670 con 0x7fe8380adeb0 2023-10-16T08:39:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83fb27640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8381527d0 con 0x7fe8380adad0 2023-10-16T08:39:17.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83e0c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8380cae10 0x7fe8380c3e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83e0c4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8380cae10 0x7fe8380c3e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44798/0 (socket says 172.21.15.40:44798) 2023-10-16T08:39:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83e0c4640 1 -- 172.21.15.40:0/3147958181 learned_addr learned my addr 172.21.15.40:0/3147958181 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:17.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83d8c3640 1 --2- 172.21.15.40:0/3147958181 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8380adad0 0x7fe838153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83d0c2640 1 --2- 172.21.15.40:0/3147958181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8380adeb0 0x7fe8380c3920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:17.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83e0c4640 1 -- 172.21.15.40:0/3147958181 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8380adad0 msgr2=0x7fe838153870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83e0c4640 1 --2- 172.21.15.40:0/3147958181 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8380adad0 0x7fe838153870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.890 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83e0c4640 1 -- 172.21.15.40:0/3147958181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8380adeb0 msgr2=0x7fe8380c3920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83e0c4640 1 --2- 172.21.15.40:0/3147958181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8380adeb0 0x7fe8380c3920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:17.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83e0c4640 1 -- 172.21.15.40:0/3147958181 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe834009420 con 0x7fe8380cae10 2023-10-16T08:39:17.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.871+0000 7fe83d0c2640 1 --2- 172.21.15.40:0/3147958181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8380adeb0 0x7fe8380c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:17.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.875+0000 7fe83e0c4640 1 --2- 172.21.15.40:0/3147958181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8380cae10 0x7fe8380c3e60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe830007d40 tx=0x7fe83000d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:17.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.875+0000 7fe82e7fc640 1 -- 172.21.15.40:0/3147958181 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8300039a0 con 0x7fe8380cae10 2023-10-16T08:39:17.893 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.875+0000 7fe82e7fc640 1 -- 172.21.15.40:0/3147958181 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83000f040 con 0x7fe8380cae10 2023-10-16T08:39:17.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.875+0000 7fe83fb27640 1 -- 172.21.15.40:0/3147958181 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe83815b250 con 0x7fe8380cae10 2023-10-16T08:39:17.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.875+0000 7fe82e7fc640 1 -- 172.21.15.40:0/3147958181 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe830003d40 con 0x7fe8380cae10 2023-10-16T08:39:17.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.875+0000 7fe83fb27640 1 -- 172.21.15.40:0/3147958181 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe838152d90 con 0x7fe8380cae10 2023-10-16T08:39:17.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.875+0000 7fe83fb27640 1 -- 172.21.15.40:0/3147958181 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe838067a20 con 0x7fe8380cae10 2023-10-16T08:39:17.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.875+0000 7fe82e7fc640 1 -- 172.21.15.40:0/3147958181 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe830004030 con 0x7fe8380cae10 2023-10-16T08:39:17.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.875+0000 7fe82e7fc640 1 --2- 172.21.15.40:0/3147958181 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe81c0664a0 0x7fe81c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:17.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.875+0000 7fe82e7fc640 1 -- 172.21.15.40:0/3147958181 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8300840a0 con 0x7fe8380cae10 2023-10-16T08:39:17.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.875+0000 7fe83d8c3640 1 --2- 172.21.15.40:0/3147958181 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe81c0664a0 0x7fe81c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:17.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.875+0000 7fe83d8c3640 1 --2- 172.21.15.40:0/3147958181 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe81c0664a0 0x7fe81c068960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe8340af000 tx=0x7fe8340a7000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:17.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:17.879+0000 7fe82e7fc640 1 -- 172.21.15.40:0/3147958181 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fe830051520 con 0x7fe8380cae10 2023-10-16T08:39:18.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.039+0000 7fe83fb27640 1 -- 172.21.15.40:0/3147958181 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe838153590 con 0x7fe8380cae10 2023-10-16T08:39:18.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.039+0000 7fe82e7fc640 1 -- 172.21.15.40:0/3147958181 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe838153590 con 0x7fe8380cae10 2023-10-16T08:39:18.046 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-16T08:39:18.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.043+0000 7fe83fb27640 1 -- 172.21.15.40:0/3147958181 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe81c0664a0 msgr2=0x7fe81c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:18.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.043+0000 7fe83fb27640 1 --2- 172.21.15.40:0/3147958181 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe81c0664a0 0x7fe81c068960 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fe8340af000 tx=0x7fe8340a7000 comp rx=0 tx=0).stop 2023-10-16T08:39:18.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.043+0000 7fe83fb27640 1 -- 172.21.15.40:0/3147958181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8380cae10 msgr2=0x7fe8380c3e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:18.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.043+0000 7fe83fb27640 1 --2- 172.21.15.40:0/3147958181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8380cae10 0x7fe8380c3e60 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe830007d40 tx=0x7fe83000d4a0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.043+0000 7fe83fb27640 1 -- 172.21.15.40:0/3147958181 shutdown_connections 2023-10-16T08:39:18.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.043+0000 7fe83fb27640 1 --2- 172.21.15.40:0/3147958181 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe81c0664a0 0x7fe81c068960 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.043+0000 7fe83fb27640 1 --2- 172.21.15.40:0/3147958181 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe8380cae10 0x7fe8380c3e60 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.043+0000 7fe83fb27640 1 --2- 172.21.15.40:0/3147958181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8380adeb0 0x7fe8380c3920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.043+0000 7fe83fb27640 1 --2- 172.21.15.40:0/3147958181 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe8380adad0 0x7fe838153870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.043+0000 7fe83fb27640 1 -- 172.21.15.40:0/3147958181 >> 172.21.15.40:0/3147958181 conn(0x7fe8380ce890 msgr2=0x7fe8380c4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.043+0000 7fe83fb27640 1 -- 172.21.15.40:0/3147958181 shutdown_connections 2023-10-16T08:39:18.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.043+0000 7fe83fb27640 1 -- 172.21.15.40:0/3147958181 wait complete. 2023-10-16T08:39:18.065 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:39:18.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.191+0000 7fcdb26bf640 1 Processor -- start 2023-10-16T08:39:18.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.191+0000 7fcdb26bf640 1 -- start start 2023-10-16T08:39:18.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.191+0000 7fcdb26bf640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdac159dd0 0x7fcdac0c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:18.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.191+0000 7fcdb26bf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcdac0adec0 0x7fcdac0c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.191+0000 7fcdb26bf640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdac15b490 con 0x7fcdac06b800 2023-10-16T08:39:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.191+0000 7fcdb26bf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdac06cd70 con 0x7fcdac0adec0 2023-10-16T08:39:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.191+0000 7fcdb26bf640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac06d230 con 0x7fcdac159dd0 2023-10-16T08:39:18.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.191+0000 7fcdab7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcdac0adec0 0x7fcdac0c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:18.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.191+0000 7fcdab7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fcdac0adec0 0x7fcdac0c97f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44814/0 (socket says 172.21.15.40:44814) 2023-10-16T08:39:18.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.191+0000 7fcdab7fe640 1 -- 172.21.15.40:0/3364519526 learned_addr learned my addr 172.21.15.40:0/3364519526 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:18.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.191+0000 7fcdabfff640 1 --2- 172.21.15.40:0/3364519526 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdac159dd0 0x7fcdac0c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:18.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.191+0000 7fcdab7fe640 1 -- 172.21.15.40:0/3364519526 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdac159dd0 msgr2=0x7fcdac0c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:18.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.191+0000 7fcdab7fe640 1 --2- 172.21.15.40:0/3364519526 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdac159dd0 0x7fcdac0c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.191+0000 7fcdab7fe640 1 -- 172.21.15.40:0/3364519526 >> v1:172.21.15.86:6789/0 conn(0x7fcdac06b800 legacy=0x7fcdac0cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:18.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.191+0000 7fcdab7fe640 1 -- 172.21.15.40:0/3364519526 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdac06cf50 con 0x7fcdac0adec0 2023-10-16T08:39:18.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.195+0000 7fcdab7fe640 1 --2- 172.21.15.40:0/3364519526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac0adec0 0x7fcdac0c97f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009870 tx=0x7fcda001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7ab7ee279bb980 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.195+0000 7fcdaaffd640 1 -- 172.21.15.40:0/3364519526 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda00ab650 con 0x7fcdac0adec0 2023-10-16T08:39:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.195+0000 7fcdaaffd640 1 -- 172.21.15.40:0/3364519526 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcda00ab7b0 con 0x7fcdac0adec0 2023-10-16T08:39:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.195+0000 7fcdaaffd640 1 -- 172.21.15.40:0/3364519526 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcda00abb40 con 0x7fcdac0adec0 2023-10-16T08:39:18.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.195+0000 7fcdb26bf640 1 -- 172.21.15.40:0/3364519526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac0adec0 msgr2=0x7fcdac0c97f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.195+0000 7fcdb26bf640 1 --2- 172.21.15.40:0/3364519526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac0adec0 0x7fcdac0c97f0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fcda0009870 tx=0x7fcda001c540 comp rx=0 tx=0).stop 2023-10-16T08:39:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.195+0000 7fcdb26bf640 1 -- 172.21.15.40:0/3364519526 shutdown_connections 2023-10-16T08:39:18.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.195+0000 7fcdb26bf640 1 --2- 172.21.15.40:0/3364519526 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac0adec0 0x7fcdac0c97f0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.195+0000 7fcdb26bf640 1 --2- 172.21.15.40:0/3364519526 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdac159dd0 0x7fcdac0c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.195+0000 7fcdb26bf640 1 -- 172.21.15.40:0/3364519526 >> 172.21.15.40:0/3364519526 conn(0x7fcdac0ce890 msgr2=0x7fcdac0ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:18.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.195+0000 7fcdb26bf640 1 -- 172.21.15.40:0/3364519526 shutdown_connections 2023-10-16T08:39:18.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.195+0000 7fcdb26bf640 1 -- 172.21.15.40:0/3364519526 wait complete. 2023-10-16T08:39:18.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.195+0000 7fcdb26bf640 1 Processor -- start 2023-10-16T08:39:18.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.195+0000 7fcdb26bf640 1 -- start start 2023-10-16T08:39:18.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdb26bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac06b800 0x7fcdac148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:18.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdb26bf640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdac0adec0 0x7fcdac0acb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:18.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdb26bf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac159dd0 0x7fcdac070a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:18.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdb26bf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac1fa470 con 0x7fcdac06b800 2023-10-16T08:39:18.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdb26bf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac0ad370 con 0x7fcdac159dd0 2023-10-16T08:39:18.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdb26bf640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdac1f9fe0 con 0x7fcdac0adec0 2023-10-16T08:39:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdabfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac06b800 0x7fcdac148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdabfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac06b800 0x7fcdac148480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44816/0 (socket says 172.21.15.40:44816) 2023-10-16T08:39:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdabfff640 1 -- 172.21.15.40:0/2914028781 learned_addr learned my addr 172.21.15.40:0/2914028781 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:18.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdab7fe640 1 --2- 172.21.15.40:0/2914028781 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdac0adec0 0x7fcdac0acb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdb0c5c640 1 --2- 172.21.15.40:0/2914028781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac159dd0 0x7fcdac070a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdabfff640 1 -- 172.21.15.40:0/2914028781 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdac0adec0 msgr2=0x7fcdac0acb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:18.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdabfff640 1 --2- 172.21.15.40:0/2914028781 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdac0adec0 0x7fcdac0acb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdabfff640 1 -- 172.21.15.40:0/2914028781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac159dd0 msgr2=0x7fcdac070a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdabfff640 1 --2- 172.21.15.40:0/2914028781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac159dd0 0x7fcdac070a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdabfff640 1 -- 172.21.15.40:0/2914028781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcda00094d0 con 0x7fcdac06b800 2023-10-16T08:39:18.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdabfff640 1 --2- 172.21.15.40:0/2914028781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac06b800 0x7fcdac148480 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00d5f0 tx=0x7fcd9c00dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:18.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcda8ff9640 1 -- 172.21.15.40:0/2914028781 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c0125f0 con 0x7fcdac06b800 2023-10-16T08:39:18.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcda8ff9640 1 -- 172.21.15.40:0/2914028781 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd9c012750 con 0x7fcdac06b800 2023-10-16T08:39:18.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdb26bf640 1 -- 172.21.15.40:0/2914028781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdac1f96b0 con 0x7fcdac06b800 2023-10-16T08:39:18.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcda8ff9640 1 -- 172.21.15.40:0/2914028781 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd9c012ae0 con 0x7fcdac06b800 2023-10-16T08:39:18.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdb26bf640 1 -- 172.21.15.40:0/2914028781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdac1f9c70 con 0x7fcdac06b800 2023-10-16T08:39:18.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdb26bf640 1 -- 172.21.15.40:0/2914028781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdac067a20 con 0x7fcdac06b800 2023-10-16T08:39:18.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcda8ff9640 1 -- 172.21.15.40:0/2914028781 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fcd9c012c40 con 0x7fcdac06b800 2023-10-16T08:39:18.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcda8ff9640 1 --2- 172.21.15.40:0/2914028781 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fcd980664a0 0x7fcd98068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:18.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcda8ff9640 1 -- 172.21.15.40:0/2914028781 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd9c0843c0 con 0x7fcdac06b800 2023-10-16T08:39:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.199+0000 7fcdab7fe640 1 --2- 172.21.15.40:0/2914028781 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fcd980664a0 0x7fcd98068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.203+0000 7fcdab7fe640 1 --2- 172.21.15.40:0/2914028781 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fcd980664a0 0x7fcd98068960 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcda00af000 tx=0x7fcda00a7000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:18.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.203+0000 7fcda8ff9640 1 -- 172.21.15.40:0/2914028781 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fcd9c051830 con 0x7fcdac06b800 2023-10-16T08:39:18.369 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.363+0000 7fcdb26bf640 1 -- 172.21.15.40:0/2914028781 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcdac0ce690 con 0x7fcdac06b800 2023-10-16T08:39:18.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.363+0000 7fcda8ff9640 1 -- 172.21.15.40:0/2914028781 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcd9c054e50 con 0x7fcdac06b800 2023-10-16T08:39:18.370 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-16T08:39:18.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.367+0000 7fcdb26bf640 1 -- 172.21.15.40:0/2914028781 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fcd980664a0 msgr2=0x7fcd98068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:18.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.367+0000 7fcdb26bf640 1 --2- 172.21.15.40:0/2914028781 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fcd980664a0 0x7fcd98068960 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fcda00af000 tx=0x7fcda00a7000 comp rx=0 tx=0).stop 2023-10-16T08:39:18.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.367+0000 7fcdb26bf640 1 -- 172.21.15.40:0/2914028781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac06b800 msgr2=0x7fcdac148480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:18.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.367+0000 7fcdb26bf640 1 --2- 172.21.15.40:0/2914028781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac06b800 0x7fcdac148480 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fcd9c00d5f0 tx=0x7fcd9c00dab0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.367+0000 7fcdb26bf640 1 -- 172.21.15.40:0/2914028781 shutdown_connections 2023-10-16T08:39:18.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.367+0000 7fcdb26bf640 1 --2- 172.21.15.40:0/2914028781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcdac159dd0 0x7fcdac070a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.367+0000 7fcdb26bf640 1 --2- 172.21.15.40:0/2914028781 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fcdac0adec0 0x7fcdac0acb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.367+0000 7fcdb26bf640 1 --2- 172.21.15.40:0/2914028781 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fcd980664a0 0x7fcd98068960 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.367+0000 7fcdb26bf640 1 --2- 172.21.15.40:0/2914028781 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fcdac06b800 0x7fcdac148480 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.367+0000 7fcdb26bf640 1 -- 172.21.15.40:0/2914028781 >> 172.21.15.40:0/2914028781 conn(0x7fcdac0ce890 msgr2=0x7fcdac0ccf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:18.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.367+0000 7fcdb26bf640 1 -- 172.21.15.40:0/2914028781 shutdown_connections 2023-10-16T08:39:18.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.367+0000 7fcdb26bf640 1 -- 172.21.15.40:0/2914028781 wait complete. 2023-10-16T08:39:18.389 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:39:18.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.515+0000 7f8448ae5640 1 Processor -- start 2023-10-16T08:39:18.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.515+0000 7f8448ae5640 1 -- start start 2023-10-16T08:39:18.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.515+0000 7f8448ae5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f84440cae10 0x7f84440b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:18.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.515+0000 7f8448ae5640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84440c98b0 con 0x7f84440adeb0 2023-10-16T08:39:18.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.515+0000 7f8448ae5640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84440b47c0 con 0x7f84440adad0 2023-10-16T08:39:18.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.515+0000 7f8448ae5640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84440c8f80 con 0x7f84440cae10 2023-10-16T08:39:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.515+0000 7f8441d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f84440cae10 0x7f84440b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.515+0000 7f8442d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f84440adeb0 0x7f844415a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60900/0 (socket says 172.21.15.40:60900) 2023-10-16T08:39:18.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.515+0000 7f8441d9b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f84440cae10 0x7f84440b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44826/0 (socket says 172.21.15.40:44826) 2023-10-16T08:39:18.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.515+0000 7f8442d9d640 1 -- 172.21.15.40:0/3827758042 learned_addr learned my addr 172.21.15.40:0/3827758042 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:18.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.515+0000 7f8441d9b640 1 -- 172.21.15.40:0/3827758042 >> v1:172.21.15.86:6789/0 conn(0x7f84440adad0 legacy=0x7f84440ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.515+0000 7f8441d9b640 1 -- 172.21.15.40:0/3827758042 >> v1:172.21.15.40:6789/0 conn(0x7f84440adeb0 legacy=0x7f844415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.515+0000 7f8441d9b640 1 -- 172.21.15.40:0/3827758042 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84440c9260 con 0x7f84440cae10 2023-10-16T08:39:18.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.515+0000 7f844159a640 1 -- 172.21.15.40:0/3827758042 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1710602032 0 0) 0x7f84440c98b0 con 0x7f84440adeb0 2023-10-16T08:39:18.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f8441d9b640 1 --2- 172.21.15.40:0/3827758042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84440cae10 0x7f84440b41e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8434009d90 tx=0x7f843401c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88412a023b5781e2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:18.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f844159a640 1 -- 172.21.15.40:0/3827758042 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84340a5a00 con 0x7f84440cae10 2023-10-16T08:39:18.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f844159a640 1 -- 172.21.15.40:0/3827758042 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84340a5b60 con 0x7f84440cae10 2023-10-16T08:39:18.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f844159a640 1 -- 172.21.15.40:0/3827758042 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84340ad440 con 0x7f84440cae10 2023-10-16T08:39:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f8448ae5640 1 -- 172.21.15.40:0/3827758042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84440cae10 msgr2=0x7f84440b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f8448ae5640 1 --2- 172.21.15.40:0/3827758042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84440cae10 0x7f84440b41e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8434009d90 tx=0x7f843401c920 comp rx=0 tx=0).stop 2023-10-16T08:39:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f8448ae5640 1 -- 172.21.15.40:0/3827758042 shutdown_connections 2023-10-16T08:39:18.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f8448ae5640 1 --2- 172.21.15.40:0/3827758042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84440cae10 0x7f84440b41e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f8448ae5640 1 -- 172.21.15.40:0/3827758042 >> 172.21.15.40:0/3827758042 conn(0x7f84440ce890 msgr2=0x7f84440ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f8448ae5640 1 -- 172.21.15.40:0/3827758042 shutdown_connections 2023-10-16T08:39:18.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f8448ae5640 1 -- 172.21.15.40:0/3827758042 wait complete. 2023-10-16T08:39:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f8448ae5640 1 Processor -- start 2023-10-16T08:39:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f8448ae5640 1 -- start start 2023-10-16T08:39:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f8448ae5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84440adad0 0x7f8444154ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:18.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f8448ae5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84440adeb0 0x7f84440c40a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f8448ae5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84440cae10 0x7f84441558c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f8448ae5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8444153460 con 0x7f84440adad0 2023-10-16T08:39:18.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f8448ae5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8444153740 con 0x7f84440cae10 2023-10-16T08:39:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f8448ae5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84440c38a0 con 0x7f84440adeb0 2023-10-16T08:39:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f844259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84440adad0 0x7f8444154ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f844259c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84440adad0 0x7f8444154ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44830/0 (socket says 172.21.15.40:44830) 2023-10-16T08:39:18.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f844259c640 1 -- 172.21.15.40:0/2135196187 learned_addr learned my addr 172.21.15.40:0/2135196187 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f8441d9b640 1 --2- 172.21.15.40:0/2135196187 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84440adeb0 0x7f84440c40a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.519+0000 7f8442d9d640 1 --2- 172.21.15.40:0/2135196187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84440cae10 0x7f84441558c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.523+0000 7f844259c640 1 -- 172.21.15.40:0/2135196187 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84440adeb0 msgr2=0x7f84440c40a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:18.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.523+0000 7f844259c640 1 --2- 172.21.15.40:0/2135196187 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84440adeb0 0x7f84440c40a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.523+0000 7f844259c640 1 -- 172.21.15.40:0/2135196187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84440cae10 msgr2=0x7f84441558c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.523+0000 7f844259c640 1 --2- 172.21.15.40:0/2135196187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84440cae10 0x7f84441558c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.523+0000 7f844259c640 1 -- 172.21.15.40:0/2135196187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8434009a70 con 0x7f84440adad0 2023-10-16T08:39:18.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.523+0000 7f8442d9d640 1 --2- 172.21.15.40:0/2135196187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84440cae10 0x7f84441558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:18.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.523+0000 7f844259c640 1 --2- 172.21.15.40:0/2135196187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84440adad0 0x7f8444154ef0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f842c00c500 tx=0x7f842c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:18.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.523+0000 7f843affd640 1 -- 172.21.15.40:0/2135196187 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c005220 con 0x7f84440adad0 2023-10-16T08:39:18.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.523+0000 7f843affd640 1 -- 172.21.15.40:0/2135196187 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f842c005380 con 0x7f84440adad0 2023-10-16T08:39:18.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.523+0000 7f8448ae5640 1 -- 172.21.15.40:0/2135196187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84440c3b80 con 0x7f84440adad0 2023-10-16T08:39:18.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.523+0000 7f843affd640 1 -- 172.21.15.40:0/2135196187 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f842c007270 con 0x7f84440adad0 2023-10-16T08:39:18.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.523+0000 7f8448ae5640 1 -- 172.21.15.40:0/2135196187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8444153080 con 0x7f84440adad0 2023-10-16T08:39:18.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.523+0000 7f8448ae5640 1 -- 172.21.15.40:0/2135196187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8444067a20 con 0x7f84440adad0 2023-10-16T08:39:18.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.523+0000 7f843affd640 1 -- 172.21.15.40:0/2135196187 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f842c0073d0 con 0x7f84440adad0 2023-10-16T08:39:18.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.523+0000 7f843affd640 1 --2- 172.21.15.40:0/2135196187 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f84240664d0 0x7f8424068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:18.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.523+0000 7f843affd640 1 -- 172.21.15.40:0/2135196187 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f842c083710 con 0x7f84440adad0 2023-10-16T08:39:18.541 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.523+0000 7f8441d9b640 1 --2- 172.21.15.40:0/2135196187 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f84240664d0 0x7f8424068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:18.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.523+0000 7f8441d9b640 1 --2- 172.21.15.40:0/2135196187 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f84240664d0 0x7f8424068990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f84340af000 tx=0x7f84340a7070 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:18.542 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.527+0000 7f843affd640 1 -- 172.21.15.40:0/2135196187 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f842c050ba0 con 0x7f84440adad0 2023-10-16T08:39:18.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.683+0000 7f8448ae5640 1 -- 172.21.15.40:0/2135196187 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f844404e9c0 con 0x7f84440adad0 2023-10-16T08:39:18.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.683+0000 7f843affd640 1 -- 172.21.15.40:0/2135196187 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f842c0541c0 con 0x7f84440adad0 2023-10-16T08:39:18.691 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-16T08:39:18.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.687+0000 7f8448ae5640 1 -- 172.21.15.40:0/2135196187 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f84240664d0 msgr2=0x7f8424068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:18.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.687+0000 7f8448ae5640 1 --2- 172.21.15.40:0/2135196187 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f84240664d0 0x7f8424068990 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f84340af000 tx=0x7f84340a7070 comp rx=0 tx=0).stop 2023-10-16T08:39:18.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.687+0000 7f8448ae5640 1 -- 172.21.15.40:0/2135196187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84440adad0 msgr2=0x7f8444154ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:18.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.687+0000 7f8448ae5640 1 --2- 172.21.15.40:0/2135196187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84440adad0 0x7f8444154ef0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f842c00c500 tx=0x7f842c00c9c0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.687+0000 7f8448ae5640 1 -- 172.21.15.40:0/2135196187 shutdown_connections 2023-10-16T08:39:18.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.687+0000 7f8448ae5640 1 --2- 172.21.15.40:0/2135196187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84440cae10 0x7f84441558c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.687+0000 7f8448ae5640 1 --2- 172.21.15.40:0/2135196187 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f84440adeb0 0x7f84440c40a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.687+0000 7f8448ae5640 1 --2- 172.21.15.40:0/2135196187 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f84240664d0 0x7f8424068990 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.687+0000 7f8448ae5640 1 --2- 172.21.15.40:0/2135196187 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f84440adad0 0x7f8444154ef0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.687+0000 7f8448ae5640 1 -- 172.21.15.40:0/2135196187 >> 172.21.15.40:0/2135196187 conn(0x7f84440ce890 msgr2=0x7f84440c5a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:18.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.687+0000 7f8448ae5640 1 -- 172.21.15.40:0/2135196187 shutdown_connections 2023-10-16T08:39:18.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.687+0000 7f8448ae5640 1 -- 172.21.15.40:0/2135196187 wait complete. 2023-10-16T08:39:18.709 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:39:18.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.831+0000 7fb681720640 1 Processor -- start 2023-10-16T08:39:18.836 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.831+0000 7fb681720640 1 -- start start 2023-10-16T08:39:18.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.831+0000 7fb681720640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb67c0b7440 0x7fb67c0d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:18.837 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.831+0000 7fb681720640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67c0d3300 con 0x7fb67c160510 2023-10-16T08:39:18.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.831+0000 7fb681720640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67c0d35e0 con 0x7fb67c0d0310 2023-10-16T08:39:18.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.831+0000 7fb681720640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb67c0d38c0 con 0x7fb67c0b7440 2023-10-16T08:39:18.838 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.831+0000 7fb67b7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb67c0b7440 0x7fb67c0d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:18.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.831+0000 7fb67affd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb67c160510 0x7fb67c0cfc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60908/0 (socket says 172.21.15.40:60908) 2023-10-16T08:39:18.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.831+0000 7fb67affd640 1 -- 172.21.15.40:0/1468863940 learned_addr learned my addr 172.21.15.40:0/1468863940 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:18.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.831+0000 7fb67b7fe640 1 -- 172.21.15.40:0/1468863940 >> v1:172.21.15.86:6789/0 conn(0x7fb67c0d0310 legacy=0x7fb67c0b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:18.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.831+0000 7fb67b7fe640 1 -- 172.21.15.40:0/1468863940 >> v1:172.21.15.40:6789/0 conn(0x7fb67c160510 legacy=0x7fb67c0cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:18.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.831+0000 7fb67b7fe640 1 -- 172.21.15.40:0/1468863940 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c0b5f10 con 0x7fb67c0b7440 2023-10-16T08:39:18.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.835+0000 7fb67b7fe640 1 --2- 172.21.15.40:0/1468863940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c0b7440 0x7fb67c0d2c70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb674009d90 tx=0x7fb67401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7efe8f6678f8b1a3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:18.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.835+0000 7fb67a7fc640 1 -- 172.21.15.40:0/1468863940 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6740a58f0 con 0x7fb67c0b7440 2023-10-16T08:39:18.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.835+0000 7fb67a7fc640 1 -- 172.21.15.40:0/1468863940 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6740a5a50 con 0x7fb67c0b7440 2023-10-16T08:39:18.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.835+0000 7fb67a7fc640 1 -- 172.21.15.40:0/1468863940 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6740a5de0 con 0x7fb67c0b7440 2023-10-16T08:39:18.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.835+0000 7fb681720640 1 -- 172.21.15.40:0/1468863940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c0b7440 msgr2=0x7fb67c0d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:18.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.835+0000 7fb681720640 1 --2- 172.21.15.40:0/1468863940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c0b7440 0x7fb67c0d2c70 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fb674009d90 tx=0x7fb67401c910 comp rx=0 tx=0).stop 2023-10-16T08:39:18.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.835+0000 7fb681720640 1 -- 172.21.15.40:0/1468863940 shutdown_connections 2023-10-16T08:39:18.846 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.835+0000 7fb681720640 1 --2- 172.21.15.40:0/1468863940 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c0b7440 0x7fb67c0d2c70 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.835+0000 7fb681720640 1 -- 172.21.15.40:0/1468863940 >> 172.21.15.40:0/1468863940 conn(0x7fb67c06a060 msgr2=0x7fb67c06a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:18.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.835+0000 7fb681720640 1 -- 172.21.15.40:0/1468863940 shutdown_connections 2023-10-16T08:39:18.847 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.835+0000 7fb681720640 1 -- 172.21.15.40:0/1468863940 wait complete. 2023-10-16T08:39:18.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.835+0000 7fb681720640 1 Processor -- start 2023-10-16T08:39:18.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.835+0000 7fb681720640 1 -- start start 2023-10-16T08:39:18.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb681720640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb67c0b7440 0x7fb67c158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:18.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb681720640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb67c0d0310 0x7fb67c0d22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:18.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb681720640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c160510 0x7fb67c153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:18.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb681720640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c157e20 con 0x7fb67c160510 2023-10-16T08:39:18.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb681720640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c1577a0 con 0x7fb67c0b7440 2023-10-16T08:39:18.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb681720640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c154830 con 0x7fb67c0d0310 2023-10-16T08:39:18.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb67b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c160510 0x7fb67c153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:18.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb67b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c160510 0x7fb67c153e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44856/0 (socket says 172.21.15.40:44856) 2023-10-16T08:39:18.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb67b7fe640 1 -- 172.21.15.40:0/810782434 learned_addr learned my addr 172.21.15.40:0/810782434 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:18.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb67affd640 1 --2- 172.21.15.40:0/810782434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb67c0b7440 0x7fb67c158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:18.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb6727fc640 1 --2- 172.21.15.40:0/810782434 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb67c0d0310 0x7fb67c0d22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:18.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb67b7fe640 1 -- 172.21.15.40:0/810782434 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb67c0d0310 msgr2=0x7fb67c0d22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:18.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb67b7fe640 1 --2- 172.21.15.40:0/810782434 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb67c0d0310 0x7fb67c0d22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb67b7fe640 1 -- 172.21.15.40:0/810782434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb67c0b7440 msgr2=0x7fb67c158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:18.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb67b7fe640 1 --2- 172.21.15.40:0/810782434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb67c0b7440 0x7fb67c158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:18.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb67b7fe640 1 -- 172.21.15.40:0/810782434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb674009a70 con 0x7fb67c160510 2023-10-16T08:39:18.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb67b7fe640 1 --2- 172.21.15.40:0/810782434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c160510 0x7fb67c153e50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb6740b0040 tx=0x7fb6740196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:18.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb673fff640 1 -- 172.21.15.40:0/810782434 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6740b9860 con 0x7fb67c160510 2023-10-16T08:39:18.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb673fff640 1 -- 172.21.15.40:0/810782434 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6740b99c0 con 0x7fb67c160510 2023-10-16T08:39:18.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb681720640 1 -- 172.21.15.40:0/810782434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67c157a40 con 0x7fb67c160510 2023-10-16T08:39:18.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb673fff640 1 -- 172.21.15.40:0/810782434 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6740b9d70 con 0x7fb67c160510 2023-10-16T08:39:18.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb681720640 1 -- 172.21.15.40:0/810782434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67c158780 con 0x7fb67c160510 2023-10-16T08:39:18.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb681720640 1 -- 172.21.15.40:0/810782434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67c158910 con 0x7fb67c160510 2023-10-16T08:39:18.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb673fff640 1 -- 172.21.15.40:0/810782434 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb6740ad440 con 0x7fb67c160510 2023-10-16T08:39:18.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb673fff640 1 --2- 172.21.15.40:0/810782434 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb65c0664a0 0x7fb65c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:18.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.839+0000 7fb673fff640 1 -- 172.21.15.40:0/810782434 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb67411f420 con 0x7fb67c160510 2023-10-16T08:39:18.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.843+0000 7fb67affd640 1 --2- 172.21.15.40:0/810782434 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb65c0664a0 0x7fb65c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:18.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.843+0000 7fb67affd640 1 --2- 172.21.15.40:0/810782434 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb65c0664a0 0x7fb65c068960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb66c005890 tx=0x7fb66c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:18.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:18.847+0000 7fb673fff640 1 -- 172.21.15.40:0/810782434 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fb6740ec890 con 0x7fb67c160510 2023-10-16T08:39:19.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.007+0000 7fb681720640 1 -- 172.21.15.40:0/810782434 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb67c157600 con 0x7fb67c160510 2023-10-16T08:39:19.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.007+0000 7fb673fff640 1 -- 172.21.15.40:0/810782434 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fb67c157600 con 0x7fb67c160510 2023-10-16T08:39:19.014 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-16T08:39:19.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.011+0000 7fb681720640 1 -- 172.21.15.40:0/810782434 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb65c0664a0 msgr2=0x7fb65c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:19.016 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.011+0000 7fb681720640 1 --2- 172.21.15.40:0/810782434 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb65c0664a0 0x7fb65c068960 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb66c005890 tx=0x7fb66c009040 comp rx=0 tx=0).stop 2023-10-16T08:39:19.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.011+0000 7fb681720640 1 -- 172.21.15.40:0/810782434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c160510 msgr2=0x7fb67c153e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:19.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.011+0000 7fb681720640 1 --2- 172.21.15.40:0/810782434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c160510 0x7fb67c153e50 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb6740b0040 tx=0x7fb6740196b0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.017 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.011+0000 7fb681720640 1 -- 172.21.15.40:0/810782434 shutdown_connections 2023-10-16T08:39:19.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.011+0000 7fb681720640 1 --2- 172.21.15.40:0/810782434 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb65c0664a0 0x7fb65c068960 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.011+0000 7fb681720640 1 --2- 172.21.15.40:0/810782434 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb67c160510 0x7fb67c153e50 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.011+0000 7fb681720640 1 --2- 172.21.15.40:0/810782434 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb67c0d0310 0x7fb67c0d22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.018 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.011+0000 7fb681720640 1 --2- 172.21.15.40:0/810782434 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb67c0b7440 0x7fb67c158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.011+0000 7fb681720640 1 -- 172.21.15.40:0/810782434 >> 172.21.15.40:0/810782434 conn(0x7fb67c06a060 msgr2=0x7fb67c06a440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:19.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.011+0000 7fb681720640 1 -- 172.21.15.40:0/810782434 shutdown_connections 2023-10-16T08:39:19.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.011+0000 7fb681720640 1 -- 172.21.15.40:0/810782434 wait complete. 2023-10-16T08:39:19.032 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi040.front.sepia.ceph.com... 2023-10-16T08:39:19.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:19.033 DEBUG:teuthology.orchestra.run.smithi040:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-16T08:39:19.053 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi040.stderr:ceph-fuse[45077]: fuse finished with error 0 and tester_r 0 2023-10-16T08:39:19.054 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:39:19.140 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-10-16T08:39:19.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:19.141 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-16T08:39:19.156 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:39:19.157 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:39:19.203 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:39:19.204 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:39:19.255 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:39:19.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.423+0000 7f0706449640 1 Processor -- start 2023-10-16T08:39:19.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.423+0000 7f0706449640 1 -- start start 2023-10-16T08:39:19.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.423+0000 7f0706449640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f07000ce3b0 0x7f070006b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:19.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.423+0000 7f0706449640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0700165c90 con 0x7f07001648d0 2023-10-16T08:39:19.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.423+0000 7f0706449640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0700165080 con 0x7f07000c15f0 2023-10-16T08:39:19.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.423+0000 7f0706449640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0700165360 con 0x7f07000ce3b0 2023-10-16T08:39:19.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.423+0000 7f06ff7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f07000ce3b0 0x7f070006b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:19.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.423+0000 7f06fffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f07001648d0 0x7f07000cdca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60922/0 (socket says 172.21.15.40:60922) 2023-10-16T08:39:19.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.423+0000 7f06fffff640 1 -- 172.21.15.40:0/2546716981 learned_addr learned my addr 172.21.15.40:0/2546716981 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:19.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.423+0000 7f06feffd640 1 -- 172.21.15.40:0/2546716981 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3203162369 0 0) 0x7f0700165c90 con 0x7f07001648d0 2023-10-16T08:39:19.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.423+0000 7f06feffd640 1 -- 172.21.15.40:0/2546716981 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f06ec003530 con 0x7f07001648d0 2023-10-16T08:39:19.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.423+0000 7f06ff7fe640 1 -- 172.21.15.40:0/2546716981 >> v1:172.21.15.86:6789/0 conn(0x7f07000c15f0 legacy=0x7f070006ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:19.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.423+0000 7f06ff7fe640 1 -- 172.21.15.40:0/2546716981 >> v1:172.21.15.40:6789/0 conn(0x7f07001648d0 legacy=0x7f07000cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:19.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.423+0000 7f06ff7fe640 1 -- 172.21.15.40:0/2546716981 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0700165640 con 0x7f07000ce3b0 2023-10-16T08:39:19.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f06ff7fe640 1 --2- 172.21.15.40:0/2546716981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000ce3b0 0x7f070006b540 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f06e80097c0 tx=0x7f070006c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4749ff8779709a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:19.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f06feffd640 1 -- 172.21.15.40:0/2546716981 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e80ab600 con 0x7f07000ce3b0 2023-10-16T08:39:19.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f06feffd640 1 -- 172.21.15.40:0/2546716981 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e80a7050 con 0x7f07000ce3b0 2023-10-16T08:39:19.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f06feffd640 1 -- 172.21.15.40:0/2546716981 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e80ab980 con 0x7f07000ce3b0 2023-10-16T08:39:19.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f0706449640 1 -- 172.21.15.40:0/2546716981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000ce3b0 msgr2=0x7f070006b540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f0706449640 1 --2- 172.21.15.40:0/2546716981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000ce3b0 0x7f070006b540 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f06e80097c0 tx=0x7f070006c9a0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f0706449640 1 -- 172.21.15.40:0/2546716981 shutdown_connections 2023-10-16T08:39:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f0706449640 1 --2- 172.21.15.40:0/2546716981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000ce3b0 0x7f070006b540 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f0706449640 1 -- 172.21.15.40:0/2546716981 >> 172.21.15.40:0/2546716981 conn(0x7f07000ce960 msgr2=0x7f07000ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:19.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f0706449640 1 -- 172.21.15.40:0/2546716981 shutdown_connections 2023-10-16T08:39:19.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f0706449640 1 -- 172.21.15.40:0/2546716981 wait complete. 2023-10-16T08:39:19.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f0706449640 1 Processor -- start 2023-10-16T08:39:19.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f0706449640 1 -- start start 2023-10-16T08:39:19.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f0706449640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07000c15f0 0x7f07000c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:19.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f0706449640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000ce3b0 0x7f07000c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:19.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f0706449640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07001648d0 0x7f070006ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:19.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f0706449640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07000c6020 con 0x7f07000ce3b0 2023-10-16T08:39:19.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f0706449640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07000b1720 con 0x7f07000c15f0 2023-10-16T08:39:19.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f0706449640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07000b1a00 con 0x7f07001648d0 2023-10-16T08:39:19.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f06ff7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000ce3b0 0x7f07000c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:19.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f06ff7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000ce3b0 0x7f07000c4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44874/0 (socket says 172.21.15.40:44874) 2023-10-16T08:39:19.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.427+0000 7f06ff7fe640 1 -- 172.21.15.40:0/3653409451 learned_addr learned my addr 172.21.15.40:0/3653409451 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:19.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.431+0000 7f06fffff640 1 --2- 172.21.15.40:0/3653409451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07000c15f0 0x7f07000c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:19.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.431+0000 7f07049e6640 1 --2- 172.21.15.40:0/3653409451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07001648d0 0x7f070006ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:19.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.431+0000 7f06ff7fe640 1 -- 172.21.15.40:0/3653409451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07001648d0 msgr2=0x7f070006ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:19.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.431+0000 7f06ff7fe640 1 --2- 172.21.15.40:0/3653409451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07001648d0 0x7f070006ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.431+0000 7f06ff7fe640 1 -- 172.21.15.40:0/3653409451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07000c15f0 msgr2=0x7f07000c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:19.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.431+0000 7f06ff7fe640 1 --2- 172.21.15.40:0/3653409451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07000c15f0 0x7f07000c5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.431+0000 7f06ff7fe640 1 -- 172.21.15.40:0/3653409451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06e8009420 con 0x7f07000ce3b0 2023-10-16T08:39:19.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.431+0000 7f06fffff640 1 --2- 172.21.15.40:0/3653409451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07000c15f0 0x7f07000c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:19.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.431+0000 7f07049e6640 1 --2- 172.21.15.40:0/3653409451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07001648d0 0x7f070006ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:19.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.431+0000 7f06ff7fe640 1 --2- 172.21.15.40:0/3653409451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000ce3b0 0x7f07000c4b20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f06e80ae040 tx=0x7f06e80190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:19.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.431+0000 7f06fcff9640 1 -- 172.21.15.40:0/3653409451 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e80b4430 con 0x7f07000ce3b0 2023-10-16T08:39:19.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.431+0000 7f06fcff9640 1 -- 172.21.15.40:0/3653409451 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06e80b4590 con 0x7f07000ce3b0 2023-10-16T08:39:19.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.431+0000 7f0706449640 1 -- 172.21.15.40:0/3653409451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07000b1ce0 con 0x7f07000ce3b0 2023-10-16T08:39:19.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.431+0000 7f06fcff9640 1 -- 172.21.15.40:0/3653409451 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06e80b4940 con 0x7f07000ce3b0 2023-10-16T08:39:19.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.431+0000 7f0706449640 1 -- 172.21.15.40:0/3653409451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07000c5060 con 0x7f07000ce3b0 2023-10-16T08:39:19.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.431+0000 7f0706449640 1 -- 172.21.15.40:0/3653409451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07000b2200 con 0x7f07000ce3b0 2023-10-16T08:39:19.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.431+0000 7f06fcff9640 1 -- 172.21.15.40:0/3653409451 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f06e80b2040 con 0x7f07000ce3b0 2023-10-16T08:39:19.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.431+0000 7f06fcff9640 1 --2- 172.21.15.40:0/3653409451 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f06e00664a0 0x7f06e0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:19.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.431+0000 7f06fcff9640 1 -- 172.21.15.40:0/3653409451 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06e811e670 con 0x7f07000ce3b0 2023-10-16T08:39:19.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.431+0000 7f06fffff640 1 --2- 172.21.15.40:0/3653409451 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f06e00664a0 0x7f06e0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:19.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.435+0000 7f06fffff640 1 --2- 172.21.15.40:0/3653409451 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f06e00664a0 0x7f06e0068960 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f06f0001570 tx=0x7f06f0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:19.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.435+0000 7f06fcff9640 1 -- 172.21.15.40:0/3653409451 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f06e80ebcf0 con 0x7f07000ce3b0 2023-10-16T08:39:19.598 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.591+0000 7f0706449640 1 -- 172.21.15.40:0/3653409451 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07000b2200 con 0x7f07000ce3b0 2023-10-16T08:39:19.599 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.595+0000 7f06fcff9640 1 -- 172.21.15.40:0/3653409451 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f07000b2200 con 0x7f07000ce3b0 2023-10-16T08:39:19.600 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:39:19.600 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:46.611098+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:39:19.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.595+0000 7f0706449640 1 -- 172.21.15.40:0/3653409451 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f06e00664a0 msgr2=0x7f06e0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:19.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.595+0000 7f0706449640 1 --2- 172.21.15.40:0/3653409451 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f06e00664a0 0x7f06e0068960 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f06f0001570 tx=0x7f06f0009040 comp rx=0 tx=0).stop 2023-10-16T08:39:19.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.595+0000 7f0706449640 1 -- 172.21.15.40:0/3653409451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000ce3b0 msgr2=0x7f07000c4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:19.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.595+0000 7f0706449640 1 --2- 172.21.15.40:0/3653409451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000ce3b0 0x7f07000c4b20 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f06e80ae040 tx=0x7f06e80190d0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.595+0000 7f0706449640 1 -- 172.21.15.40:0/3653409451 shutdown_connections 2023-10-16T08:39:19.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.595+0000 7f0706449640 1 --2- 172.21.15.40:0/3653409451 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f07001648d0 0x7f070006ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.595+0000 7f0706449640 1 --2- 172.21.15.40:0/3653409451 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f06e00664a0 0x7f06e0068960 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.595+0000 7f0706449640 1 --2- 172.21.15.40:0/3653409451 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f07000ce3b0 0x7f07000c4b20 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.595+0000 7f0706449640 1 --2- 172.21.15.40:0/3653409451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07000c15f0 0x7f07000c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.595+0000 7f0706449640 1 -- 172.21.15.40:0/3653409451 >> 172.21.15.40:0/3653409451 conn(0x7f07000ce960 msgr2=0x7f07000ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:19.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.595+0000 7f0706449640 1 -- 172.21.15.40:0/3653409451 shutdown_connections 2023-10-16T08:39:19.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.595+0000 7f0706449640 1 -- 172.21.15.40:0/3653409451 wait complete. 2023-10-16T08:39:19.619 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:38:33.014668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:38:43.055215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:38:43.971666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:39:19.620 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:39:19.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.735+0000 7f2ab90a1640 1 Processor -- start 2023-10-16T08:39:19.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.735+0000 7f2ab90a1640 1 -- start start 2023-10-16T08:39:19.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab90a1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ab415d030 0x7f2ab406aab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:19.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab90a1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab406a030 con 0x7f2ab415cc50 2023-10-16T08:39:19.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab90a1640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab406bc40 con 0x7f2ab40afed0 2023-10-16T08:39:19.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab90a1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab406bf20 con 0x7f2ab415d030 2023-10-16T08:39:19.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab2d9d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f2ab415cc50 0x7f2ab406b480 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60924/0 (socket says 172.21.15.40:60924) 2023-10-16T08:39:19.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab2d9d640 1 -- 172.21.15.40:0/2477999232 learned_addr learned my addr 172.21.15.40:0/2477999232 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:19.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab259c640 1 --2- 172.21.15.40:0/2477999232 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ab415d030 0x7f2ab406aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:19.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab1d9b640 1 -- 172.21.15.40:0/2477999232 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3330426717 0 0) 0x7f2ab406a030 con 0x7f2ab415cc50 2023-10-16T08:39:19.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab1d9b640 1 -- 172.21.15.40:0/2477999232 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2aa0003530 con 0x7f2ab415cc50 2023-10-16T08:39:19.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab1d9b640 1 -- 172.21.15.40:0/2477999232 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1179212974 0 0) 0x7f2aa0003530 con 0x7f2ab415cc50 2023-10-16T08:39:19.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab1d9b640 1 -- 172.21.15.40:0/2477999232 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ab415d030 msgr2=0x7f2ab406aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:19.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab1d9b640 1 --2- 172.21.15.40:0/2477999232 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ab415d030 0x7f2ab406aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab1d9b640 1 -- 172.21.15.40:0/2477999232 >> v1:172.21.15.86:6789/0 conn(0x7f2ab40afed0 legacy=0x7f2ab4071a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:19.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab1d9b640 1 -- 172.21.15.40:0/2477999232 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab406a2d0 con 0x7f2ab415cc50 2023-10-16T08:39:19.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab1d9b640 1 -- 172.21.15.40:0/2477999232 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f2aa4001a00 con 0x7f2ab415cc50 2023-10-16T08:39:19.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab1d9b640 1 -- 172.21.15.40:0/2477999232 >> v1:172.21.15.40:6789/0 conn(0x7f2ab415cc50 legacy=0x7f2ab406b480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:19.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab1d9b640 1 --2- 172.21.15.40:0/2477999232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2aa0003a20 0x7f2aa0003e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:19.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab1d9b640 1 --2- 172.21.15.40:0/2477999232 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ab415d030 0x7f2aa00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:19.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab1d9b640 1 --2- 172.21.15.40:0/2477999232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aa0004d10 0x7f2aa0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:19.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab1d9b640 1 -- 172.21.15.40:0/2477999232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab406bf20 con 0x7f2aa0003a20 2023-10-16T08:39:19.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab1d9b640 1 -- 172.21.15.40:0/2477999232 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa000ba40 con 0x7f2aa0004d10 2023-10-16T08:39:19.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab1d9b640 1 -- 172.21.15.40:0/2477999232 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2aa000bc00 con 0x7f2ab415d030 2023-10-16T08:39:19.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab2d9d640 1 --2- 172.21.15.40:0/2477999232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2aa0003a20 0x7f2aa0003e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:19.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab259c640 1 --2- 172.21.15.40:0/2477999232 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ab415d030 0x7f2aa00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:19.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab359e640 1 --2- 172.21.15.40:0/2477999232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aa0004d10 0x7f2aa0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:19.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab2d9d640 1 -- 172.21.15.40:0/2477999232 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ab415d030 msgr2=0x7f2aa00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:19.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab2d9d640 1 --2- 172.21.15.40:0/2477999232 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ab415d030 0x7f2aa00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab2d9d640 1 -- 172.21.15.40:0/2477999232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aa0004d10 msgr2=0x7f2aa0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:19.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab2d9d640 1 --2- 172.21.15.40:0/2477999232 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aa0004d10 0x7f2aa0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.739+0000 7f2ab2d9d640 1 -- 172.21.15.40:0/2477999232 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aa0003530 con 0x7f2aa0003a20 2023-10-16T08:39:19.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab2d9d640 1 --2- 172.21.15.40:0/2477999232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2aa0003a20 0x7f2aa0003e00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400a840 tx=0x7f2aa401d310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6a1be0a6f040d3e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:19.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab1d9b640 1 -- 172.21.15.40:0/2477999232 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa40ad630 con 0x7f2aa0003a20 2023-10-16T08:39:19.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab1d9b640 1 -- 172.21.15.40:0/2477999232 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aa40a8070 con 0x7f2aa0003a20 2023-10-16T08:39:19.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab1d9b640 1 -- 172.21.15.40:0/2477999232 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aa40ad9d0 con 0x7f2aa0003a20 2023-10-16T08:39:19.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab90a1640 1 -- 172.21.15.40:0/2477999232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2aa0003a20 msgr2=0x7f2aa0003e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:19.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab90a1640 1 --2- 172.21.15.40:0/2477999232 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2aa0003a20 0x7f2aa0003e00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2aa400a840 tx=0x7f2aa401d310 comp rx=0 tx=0).stop 2023-10-16T08:39:19.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab359e640 1 -- 172.21.15.40:0/2477999232 reap_dead start 2023-10-16T08:39:19.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab90a1640 1 -- 172.21.15.40:0/2477999232 shutdown_connections 2023-10-16T08:39:19.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab90a1640 1 -- 172.21.15.40:0/2477999232 >> 172.21.15.40:0/2477999232 conn(0x7f2ab4065e80 msgr2=0x7f2ab4066290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:19.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab90a1640 1 -- 172.21.15.40:0/2477999232 shutdown_connections 2023-10-16T08:39:19.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab90a1640 1 -- 172.21.15.40:0/2477999232 wait complete. 2023-10-16T08:39:19.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab90a1640 1 Processor -- start 2023-10-16T08:39:19.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab90a1640 1 -- start start 2023-10-16T08:39:19.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab90a1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ab4159fb0 0x7f2ab40ae990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:19.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab90a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab40ae4d0 0x7f2ab40aeed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:19.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab90a1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab415a390 0x7f2ab40acc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:19.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab90a1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab415ed40 con 0x7f2ab415a390 2023-10-16T08:39:19.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab90a1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab40af750 con 0x7f2ab40ae4d0 2023-10-16T08:39:19.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab90a1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab415de50 con 0x7f2ab4159fb0 2023-10-16T08:39:19.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab415a390 0x7f2ab40acc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:19.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab359e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab415a390 0x7f2ab40acc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44882/0 (socket says 172.21.15.40:44882) 2023-10-16T08:39:19.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab359e640 1 -- 172.21.15.40:0/1632785636 learned_addr learned my addr 172.21.15.40:0/1632785636 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:19.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab259c640 1 --2- 172.21.15.40:0/1632785636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab40ae4d0 0x7f2ab40aeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:19.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab2d9d640 1 --2- 172.21.15.40:0/1632785636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ab4159fb0 0x7f2ab40ae990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:19.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab359e640 1 -- 172.21.15.40:0/1632785636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ab4159fb0 msgr2=0x7f2ab40ae990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:19.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab359e640 1 --2- 172.21.15.40:0/1632785636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ab4159fb0 0x7f2ab40ae990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.762 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab359e640 1 -- 172.21.15.40:0/1632785636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab40ae4d0 msgr2=0x7f2ab40aeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab359e640 1 --2- 172.21.15.40:0/1632785636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab40ae4d0 0x7f2ab40aeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab359e640 1 -- 172.21.15.40:0/1632785636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab415e130 con 0x7f2ab415a390 2023-10-16T08:39:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.743+0000 7f2ab2d9d640 1 --2- 172.21.15.40:0/1632785636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ab4159fb0 0x7f2ab40ae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.747+0000 7f2ab359e640 1 --2- 172.21.15.40:0/1632785636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab415a390 0x7f2ab40acc70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40ce870 tx=0x7f2aac00e760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:19.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.747+0000 7f2aab7fe640 1 -- 172.21.15.40:0/1632785636 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac0052e0 con 0x7f2ab415a390 2023-10-16T08:39:19.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.747+0000 7f2aab7fe640 1 -- 172.21.15.40:0/1632785636 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2aac005480 con 0x7f2ab415a390 2023-10-16T08:39:19.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.747+0000 7f2aab7fe640 1 -- 172.21.15.40:0/1632785636 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2aac003840 con 0x7f2ab415a390 2023-10-16T08:39:19.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.747+0000 7f2ab90a1640 1 -- 172.21.15.40:0/1632785636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab415e410 con 0x7f2ab415a390 2023-10-16T08:39:19.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.747+0000 7f2ab90a1640 1 -- 172.21.15.40:0/1632785636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab415e9d0 con 0x7f2ab415a390 2023-10-16T08:39:19.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.747+0000 7f2ab90a1640 1 -- 172.21.15.40:0/1632785636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab404e9c0 con 0x7f2ab415a390 2023-10-16T08:39:19.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.747+0000 7f2aab7fe640 1 -- 172.21.15.40:0/1632785636 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f2aac0039a0 con 0x7f2ab415a390 2023-10-16T08:39:19.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.747+0000 7f2aab7fe640 1 --2- 172.21.15.40:0/1632785636 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f2a940664d0 0x7f2a94068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:19.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.747+0000 7f2aab7fe640 1 -- 172.21.15.40:0/1632785636 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2aac0833c0 con 0x7f2ab415a390 2023-10-16T08:39:19.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.747+0000 7f2ab2d9d640 1 --2- 172.21.15.40:0/1632785636 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f2a940664d0 0x7f2a94068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:19.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.747+0000 7f2ab2d9d640 1 --2- 172.21.15.40:0/1632785636 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f2a940664d0 0x7f2a94068990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40ad000 tx=0x7f2aa4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:19.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.751+0000 7f2aab7fe640 1 -- 172.21.15.40:0/1632785636 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f2aac0508f0 con 0x7f2ab415a390 2023-10-16T08:39:19.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.907+0000 7f2ab90a1640 1 -- 172.21.15.40:0/1632785636 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2ab40ccbd0 con 0x7f2ab415a390 2023-10-16T08:39:19.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.911+0000 7f2aab7fe640 1 -- 172.21.15.40:0/1632785636 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2aac053f10 con 0x7f2ab415a390 2023-10-16T08:39:19.915 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-16T08:39:19.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.911+0000 7f2ab90a1640 1 -- 172.21.15.40:0/1632785636 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f2a940664d0 msgr2=0x7f2a94068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:19.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.911+0000 7f2ab90a1640 1 --2- 172.21.15.40:0/1632785636 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f2a940664d0 0x7f2a94068990 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2aa40ad000 tx=0x7f2aa4009040 comp rx=0 tx=0).stop 2023-10-16T08:39:19.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.911+0000 7f2ab90a1640 1 -- 172.21.15.40:0/1632785636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab415a390 msgr2=0x7f2ab40acc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:19.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.911+0000 7f2ab90a1640 1 --2- 172.21.15.40:0/1632785636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab415a390 0x7f2ab40acc70 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f2ab40ce870 tx=0x7f2aac00e760 comp rx=0 tx=0).stop 2023-10-16T08:39:19.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.911+0000 7f2ab90a1640 1 -- 172.21.15.40:0/1632785636 shutdown_connections 2023-10-16T08:39:19.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.911+0000 7f2ab90a1640 1 --2- 172.21.15.40:0/1632785636 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f2a940664d0 0x7f2a94068990 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.911+0000 7f2ab90a1640 1 --2- 172.21.15.40:0/1632785636 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2ab415a390 0x7f2ab40acc70 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.911+0000 7f2ab90a1640 1 --2- 172.21.15.40:0/1632785636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab40ae4d0 0x7f2ab40aeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.911+0000 7f2ab90a1640 1 --2- 172.21.15.40:0/1632785636 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f2ab4159fb0 0x7f2ab40ae990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:19.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.911+0000 7f2ab90a1640 1 -- 172.21.15.40:0/1632785636 >> 172.21.15.40:0/1632785636 conn(0x7f2ab4065e80 msgr2=0x7f2ab415bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:19.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.911+0000 7f2ab90a1640 1 -- 172.21.15.40:0/1632785636 shutdown_connections 2023-10-16T08:39:19.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:19.911+0000 7f2ab90a1640 1 -- 172.21.15.40:0/1632785636 wait complete. 2023-10-16T08:39:19.935 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:39:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.059+0000 7f55b3801640 1 Processor -- start 2023-10-16T08:39:20.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.059+0000 7f55b3801640 1 -- start start 2023-10-16T08:39:20.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.059+0000 7f55b3801640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f55ac0b8590 0x7f55ac0d2c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:20.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.059+0000 7f55b3801640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f55ac0d3140 0x7f55ac0b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:20.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.059+0000 7f55b3801640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55ac0d3540 con 0x7f55ac15cdc0 2023-10-16T08:39:20.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.059+0000 7f55b3801640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55ac0d1870 con 0x7f55ac0b8590 2023-10-16T08:39:20.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.059+0000 7f55b3801640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55ac0d1b50 con 0x7f55ac0d3140 2023-10-16T08:39:20.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.059+0000 7f55b0d9c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f55ac0b8590 0x7f55ac0d2c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:20.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.059+0000 7f55b159d640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f55ac15cdc0 0x7f55ac0d1040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60932/0 (socket says 172.21.15.40:60932) 2023-10-16T08:39:20.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.059+0000 7f55b159d640 1 -- 172.21.15.40:0/3180741411 learned_addr learned my addr 172.21.15.40:0/3180741411 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:20.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.059+0000 7f55b1d9e640 1 --2- 172.21.15.40:0/3180741411 >> v2:172.21.15.86:3300/0 conn(0x7f55ac0d3140 0x7f55ac0b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:20.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.059+0000 7f55abfff640 1 -- 172.21.15.40:0/3180741411 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2381787217 0 0) 0x7f55ac0d3540 con 0x7f55ac15cdc0 2023-10-16T08:39:20.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.059+0000 7f55abfff640 1 -- 172.21.15.40:0/3180741411 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5594003530 con 0x7f55ac15cdc0 2023-10-16T08:39:20.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.059+0000 7f55b0d9c640 1 -- 172.21.15.40:0/3180741411 >> v2:172.21.15.86:3300/0 conn(0x7f55ac0d3140 msgr2=0x7f55ac0b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:20.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.059+0000 7f55b0d9c640 1 --2- 172.21.15.40:0/3180741411 >> v2:172.21.15.86:3300/0 conn(0x7f55ac0d3140 0x7f55ac0b5da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.059+0000 7f55b0d9c640 1 -- 172.21.15.40:0/3180741411 >> v1:172.21.15.40:6789/0 conn(0x7f55ac15cdc0 legacy=0x7f55ac0d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:20.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.059+0000 7f55b0d9c640 1 -- 172.21.15.40:0/3180741411 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55ac0d1e30 con 0x7f55ac0b8590 2023-10-16T08:39:20.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.059+0000 7f55b1d9e640 1 --2- 172.21.15.40:0/3180741411 >> v2:172.21.15.86:3300/0 conn(0x7f55ac0d3140 0x7f55ac0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:20.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.063+0000 7f55b0d9c640 1 --2- 172.21.15.40:0/3180741411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac0b8590 0x7f55ac0d2c00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0d0f60 tx=0x7f55a001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39f4ee7316d284fd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:20.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.063+0000 7f55abfff640 1 -- 172.21.15.40:0/3180741411 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a00a7070 con 0x7f55ac0b8590 2023-10-16T08:39:20.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.063+0000 7f55abfff640 1 -- 172.21.15.40:0/3180741411 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a00ab430 con 0x7f55ac0b8590 2023-10-16T08:39:20.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.063+0000 7f55abfff640 1 -- 172.21.15.40:0/3180741411 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a00ab7c0 con 0x7f55ac0b8590 2023-10-16T08:39:20.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.063+0000 7f55b3801640 1 -- 172.21.15.40:0/3180741411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac0b8590 msgr2=0x7f55ac0d2c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:20.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.063+0000 7f55b3801640 1 --2- 172.21.15.40:0/3180741411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac0b8590 0x7f55ac0d2c00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f55ac0d0f60 tx=0x7f55a001c430 comp rx=0 tx=0).stop 2023-10-16T08:39:20.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.063+0000 7f55b3801640 1 -- 172.21.15.40:0/3180741411 shutdown_connections 2023-10-16T08:39:20.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.063+0000 7f55b3801640 1 --2- 172.21.15.40:0/3180741411 >> v2:172.21.15.86:3300/0 conn(0x7f55ac0d3140 0x7f55ac0b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.063+0000 7f55b3801640 1 --2- 172.21.15.40:0/3180741411 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac0b8590 0x7f55ac0d2c00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.063+0000 7f55b3801640 1 -- 172.21.15.40:0/3180741411 >> 172.21.15.40:0/3180741411 conn(0x7f55ac06a290 msgr2=0x7f55ac06a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:20.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.063+0000 7f55b3801640 1 -- 172.21.15.40:0/3180741411 shutdown_connections 2023-10-16T08:39:20.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.063+0000 7f55b3801640 1 -- 172.21.15.40:0/3180741411 wait complete. 2023-10-16T08:39:20.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.063+0000 7f55b3801640 1 Processor -- start 2023-10-16T08:39:20.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.063+0000 7f55b3801640 1 -- start start 2023-10-16T08:39:20.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b3801640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55ac0b8590 0x7f55ac1495f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:20.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b3801640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac0d3140 0x7f55ac149b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:20.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b3801640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55ac15cdc0 0x7f55ac1f8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:20.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b3801640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac0c8790 con 0x7f55ac0d3140 2023-10-16T08:39:20.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b3801640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac0c8a70 con 0x7f55ac0b8590 2023-10-16T08:39:20.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b3801640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55ac0c8d50 con 0x7f55ac15cdc0 2023-10-16T08:39:20.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b0d9c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac0d3140 0x7f55ac149b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:20.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b0d9c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac0d3140 0x7f55ac149b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44906/0 (socket says 172.21.15.40:44906) 2023-10-16T08:39:20.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b0d9c640 1 -- 172.21.15.40:0/882451799 learned_addr learned my addr 172.21.15.40:0/882451799 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:20.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b159d640 1 --2- 172.21.15.40:0/882451799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55ac0b8590 0x7f55ac1495f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:20.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b1d9e640 1 --2- 172.21.15.40:0/882451799 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55ac15cdc0 0x7f55ac1f8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:20.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b0d9c640 1 -- 172.21.15.40:0/882451799 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55ac15cdc0 msgr2=0x7f55ac1f8de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:20.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b0d9c640 1 --2- 172.21.15.40:0/882451799 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55ac15cdc0 0x7f55ac1f8de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b0d9c640 1 -- 172.21.15.40:0/882451799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55ac0b8590 msgr2=0x7f55ac1495f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:20.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b0d9c640 1 --2- 172.21.15.40:0/882451799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55ac0b8590 0x7f55ac1495f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b0d9c640 1 -- 172.21.15.40:0/882451799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a0009420 con 0x7f55ac0d3140 2023-10-16T08:39:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b159d640 1 --2- 172.21.15.40:0/882451799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55ac0b8590 0x7f55ac1495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b0d9c640 1 --2- 172.21.15.40:0/882451799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac0d3140 0x7f55ac149b30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f55a0019040 tx=0x7f55a001ac70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:20.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55a9ffb640 1 -- 172.21.15.40:0/882451799 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a00b5610 con 0x7f55ac0d3140 2023-10-16T08:39:20.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55a9ffb640 1 -- 172.21.15.40:0/882451799 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55a00b5790 con 0x7f55ac0d3140 2023-10-16T08:39:20.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b3801640 1 -- 172.21.15.40:0/882451799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55ac0c9030 con 0x7f55ac0d3140 2023-10-16T08:39:20.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55a9ffb640 1 -- 172.21.15.40:0/882451799 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a00b5b40 con 0x7f55ac0d3140 2023-10-16T08:39:20.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b3801640 1 -- 172.21.15.40:0/882451799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55ac148260 con 0x7f55ac0d3140 2023-10-16T08:39:20.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55b3801640 1 -- 172.21.15.40:0/882451799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55ac148450 con 0x7f55ac0d3140 2023-10-16T08:39:20.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55a9ffb640 1 -- 172.21.15.40:0/882451799 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f55a00b5ca0 con 0x7f55ac0d3140 2023-10-16T08:39:20.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55a9ffb640 1 --2- 172.21.15.40:0/882451799 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f55980664a0 0x7f5598068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:20.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.067+0000 7f55a9ffb640 1 -- 172.21.15.40:0/882451799 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f55a011e940 con 0x7f55ac0d3140 2023-10-16T08:39:20.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.071+0000 7f55b159d640 1 --2- 172.21.15.40:0/882451799 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f55980664a0 0x7f5598068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:20.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.071+0000 7f55b159d640 1 --2- 172.21.15.40:0/882451799 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f55980664a0 0x7f5598068960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f55a4001570 tx=0x7f55a4009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:20.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.071+0000 7f55a9ffb640 1 -- 172.21.15.40:0/882451799 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f55ac148450 con 0x7f55ac0d3140 2023-10-16T08:39:20.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.231+0000 7f55b3801640 1 -- 172.21.15.40:0/882451799 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f55ac148450 con 0x7f55ac0d3140 2023-10-16T08:39:20.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.231+0000 7f55a9ffb640 1 -- 172.21.15.40:0/882451799 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f55ac148450 con 0x7f55ac0d3140 2023-10-16T08:39:20.237 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-16T08:39:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.231+0000 7f55b3801640 1 -- 172.21.15.40:0/882451799 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f55980664a0 msgr2=0x7f5598068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.231+0000 7f55b3801640 1 --2- 172.21.15.40:0/882451799 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f55980664a0 0x7f5598068960 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f55a4001570 tx=0x7f55a4009040 comp rx=0 tx=0).stop 2023-10-16T08:39:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.231+0000 7f55b3801640 1 -- 172.21.15.40:0/882451799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac0d3140 msgr2=0x7f55ac149b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:20.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.231+0000 7f55b3801640 1 --2- 172.21.15.40:0/882451799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac0d3140 0x7f55ac149b30 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f55a0019040 tx=0x7f55a001ac70 comp rx=0 tx=0).stop 2023-10-16T08:39:20.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.231+0000 7f55b3801640 1 -- 172.21.15.40:0/882451799 shutdown_connections 2023-10-16T08:39:20.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.231+0000 7f55b3801640 1 --2- 172.21.15.40:0/882451799 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f55ac15cdc0 0x7f55ac1f8de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.231+0000 7f55b3801640 1 --2- 172.21.15.40:0/882451799 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f55980664a0 0x7f5598068960 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.231+0000 7f55b3801640 1 --2- 172.21.15.40:0/882451799 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f55ac0d3140 0x7f55ac149b30 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.231+0000 7f55b3801640 1 --2- 172.21.15.40:0/882451799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f55ac0b8590 0x7f55ac1495f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.231+0000 7f55b3801640 1 -- 172.21.15.40:0/882451799 >> 172.21.15.40:0/882451799 conn(0x7f55ac06a290 msgr2=0x7f55ac15a890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:20.241 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.231+0000 7f55b3801640 1 -- 172.21.15.40:0/882451799 shutdown_connections 2023-10-16T08:39:20.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.235+0000 7f55b3801640 1 -- 172.21.15.40:0/882451799 wait complete. 2023-10-16T08:39:20.256 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:39:20.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.383+0000 7ff7eb6f9640 1 Processor -- start 2023-10-16T08:39:20.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.383+0000 7ff7eb6f9640 1 -- start start 2023-10-16T08:39:20.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.383+0000 7ff7eb6f9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff7e40cae10 0x7ff7e40b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:20.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.383+0000 7ff7eb6f9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e40c98b0 con 0x7ff7e40adeb0 2023-10-16T08:39:20.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.383+0000 7ff7eb6f9640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e40b47c0 con 0x7ff7e40adad0 2023-10-16T08:39:20.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.383+0000 7ff7eb6f9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e40c8f80 con 0x7ff7e40cae10 2023-10-16T08:39:20.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.383+0000 7ff7e8c94640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7ff7e40cae10 0x7ff7e40b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:20.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.383+0000 7ff7e9c96640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff7e40adeb0 0x7ff7e415a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60940/0 (socket says 172.21.15.40:60940) 2023-10-16T08:39:20.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.383+0000 7ff7e9c96640 1 -- 172.21.15.40:0/2282618721 learned_addr learned my addr 172.21.15.40:0/2282618721 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:20.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.383+0000 7ff7e8c94640 1 -- 172.21.15.40:0/2282618721 >> v1:172.21.15.86:6789/0 conn(0x7ff7e40adad0 legacy=0x7ff7e40ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:20.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.383+0000 7ff7e8c94640 1 -- 172.21.15.40:0/2282618721 >> v1:172.21.15.40:6789/0 conn(0x7ff7e40adeb0 legacy=0x7ff7e415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:20.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.383+0000 7ff7e8c94640 1 -- 172.21.15.40:0/2282618721 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e40c9260 con 0x7ff7e40cae10 2023-10-16T08:39:20.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7e8c94640 1 --2- 172.21.15.40:0/2282618721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e40cae10 0x7ff7e40b41e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40097c0 tx=0x7ff7d401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b522dc8932f8531 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:20.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7dbfff640 1 -- 172.21.15.40:0/2282618721 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d40ab670 con 0x7ff7e40cae10 2023-10-16T08:39:20.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7dbfff640 1 -- 172.21.15.40:0/2282618721 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d40ab7d0 con 0x7ff7e40cae10 2023-10-16T08:39:20.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7dbfff640 1 -- 172.21.15.40:0/2282618721 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d40abb60 con 0x7ff7e40cae10 2023-10-16T08:39:20.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7eb6f9640 1 -- 172.21.15.40:0/2282618721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e40cae10 msgr2=0x7ff7e40b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:20.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7eb6f9640 1 --2- 172.21.15.40:0/2282618721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e40cae10 0x7ff7e40b41e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40097c0 tx=0x7ff7d401c450 comp rx=0 tx=0).stop 2023-10-16T08:39:20.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7eb6f9640 1 -- 172.21.15.40:0/2282618721 shutdown_connections 2023-10-16T08:39:20.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7eb6f9640 1 --2- 172.21.15.40:0/2282618721 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e40cae10 0x7ff7e40b41e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7eb6f9640 1 -- 172.21.15.40:0/2282618721 >> 172.21.15.40:0/2282618721 conn(0x7ff7e40ce890 msgr2=0x7ff7e40ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:20.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7eb6f9640 1 -- 172.21.15.40:0/2282618721 shutdown_connections 2023-10-16T08:39:20.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7eb6f9640 1 -- 172.21.15.40:0/2282618721 wait complete. 2023-10-16T08:39:20.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7eb6f9640 1 Processor -- start 2023-10-16T08:39:20.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7eb6f9640 1 -- start start 2023-10-16T08:39:20.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7eb6f9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7e40adad0 0x7ff7e4151080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:20.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7eb6f9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e40adeb0 0x7ff7e40cce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:20.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7eb6f9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7e40cae10 0x7ff7e41541d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:20.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7eb6f9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e40cde10 con 0x7ff7e40adeb0 2023-10-16T08:39:20.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7eb6f9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e40cd850 con 0x7ff7e40adad0 2023-10-16T08:39:20.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7eb6f9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e40cdb30 con 0x7ff7e40cae10 2023-10-16T08:39:20.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7e8c94640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e40adeb0 0x7ff7e40cce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:20.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7e8c94640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e40adeb0 0x7ff7e40cce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44920/0 (socket says 172.21.15.40:44920) 2023-10-16T08:39:20.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7e8c94640 1 -- 172.21.15.40:0/3475506898 learned_addr learned my addr 172.21.15.40:0/3475506898 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:20.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7e9495640 1 --2- 172.21.15.40:0/3475506898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7e40adad0 0x7ff7e4151080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:20.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7e9c96640 1 --2- 172.21.15.40:0/3475506898 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7e40cae10 0x7ff7e41541d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:20.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7e8c94640 1 -- 172.21.15.40:0/3475506898 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7e40cae10 msgr2=0x7ff7e41541d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:20.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7e8c94640 1 --2- 172.21.15.40:0/3475506898 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7e40cae10 0x7ff7e41541d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7e8c94640 1 -- 172.21.15.40:0/3475506898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7e40adad0 msgr2=0x7ff7e4151080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:20.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7e8c94640 1 --2- 172.21.15.40:0/3475506898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7e40adad0 0x7ff7e4151080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7e8c94640 1 -- 172.21.15.40:0/3475506898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d4009420 con 0x7ff7e40adeb0 2023-10-16T08:39:20.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7e9495640 1 --2- 172.21.15.40:0/3475506898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7e40adad0 0x7ff7e4151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:20.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7e8c94640 1 --2- 172.21.15.40:0/3475506898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e40adeb0 0x7ff7e40cce20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40ad040 tx=0x7ff7d401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:20.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7d9ffb640 1 -- 172.21.15.40:0/3475506898 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d40b5810 con 0x7ff7e40adeb0 2023-10-16T08:39:20.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7d9ffb640 1 -- 172.21.15.40:0/3475506898 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d40b59b0 con 0x7ff7e40adeb0 2023-10-16T08:39:20.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7eb6f9640 1 -- 172.21.15.40:0/3475506898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e41539d0 con 0x7ff7e40adeb0 2023-10-16T08:39:20.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.387+0000 7ff7d9ffb640 1 -- 172.21.15.40:0/3475506898 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d40b5d60 con 0x7ff7e40adeb0 2023-10-16T08:39:20.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.391+0000 7ff7eb6f9640 1 -- 172.21.15.40:0/3475506898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e41515c0 con 0x7ff7e40adeb0 2023-10-16T08:39:20.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.391+0000 7ff7eb6f9640 1 -- 172.21.15.40:0/3475506898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7e4067a20 con 0x7ff7e40adeb0 2023-10-16T08:39:20.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.391+0000 7ff7d9ffb640 1 -- 172.21.15.40:0/3475506898 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff7d40b4390 con 0x7ff7e40adeb0 2023-10-16T08:39:20.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.391+0000 7ff7d9ffb640 1 --2- 172.21.15.40:0/3475506898 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff7c80664a0 0x7ff7c8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:20.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.391+0000 7ff7d9ffb640 1 -- 172.21.15.40:0/3475506898 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7d411eba0 con 0x7ff7e40adeb0 2023-10-16T08:39:20.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.391+0000 7ff7e9495640 1 --2- 172.21.15.40:0/3475506898 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff7c80664a0 0x7ff7c8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:20.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.391+0000 7ff7e9495640 1 --2- 172.21.15.40:0/3475506898 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff7c80664a0 0x7ff7c8068960 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4150810 tx=0x7ff7e0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:20.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.395+0000 7ff7d9ffb640 1 -- 172.21.15.40:0/3475506898 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7ff7d40ec1d0 con 0x7ff7e40adeb0 2023-10-16T08:39:20.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.551+0000 7ff7eb6f9640 1 -- 172.21.15.40:0/3475506898 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff7e404e9c0 con 0x7ff7e40adeb0 2023-10-16T08:39:20.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.551+0000 7ff7d9ffb640 1 -- 172.21.15.40:0/3475506898 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff7d40ef7f0 con 0x7ff7e40adeb0 2023-10-16T08:39:20.557 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-16T08:39:20.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.555+0000 7ff7eb6f9640 1 -- 172.21.15.40:0/3475506898 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff7c80664a0 msgr2=0x7ff7c8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:20.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.555+0000 7ff7eb6f9640 1 --2- 172.21.15.40:0/3475506898 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff7c80664a0 0x7ff7c8068960 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ff7e4150810 tx=0x7ff7e0009040 comp rx=0 tx=0).stop 2023-10-16T08:39:20.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.555+0000 7ff7eb6f9640 1 -- 172.21.15.40:0/3475506898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e40adeb0 msgr2=0x7ff7e40cce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:20.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.555+0000 7ff7eb6f9640 1 --2- 172.21.15.40:0/3475506898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e40adeb0 0x7ff7e40cce20 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7ff7d40ad040 tx=0x7ff7d401ac90 comp rx=0 tx=0).stop 2023-10-16T08:39:20.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.555+0000 7ff7eb6f9640 1 -- 172.21.15.40:0/3475506898 shutdown_connections 2023-10-16T08:39:20.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.555+0000 7ff7eb6f9640 1 --2- 172.21.15.40:0/3475506898 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff7e40cae10 0x7ff7e41541d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.555+0000 7ff7eb6f9640 1 --2- 172.21.15.40:0/3475506898 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff7c80664a0 0x7ff7c8068960 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.561 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.555+0000 7ff7eb6f9640 1 --2- 172.21.15.40:0/3475506898 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff7e40adeb0 0x7ff7e40cce20 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.555+0000 7ff7eb6f9640 1 --2- 172.21.15.40:0/3475506898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7e40adad0 0x7ff7e4151080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.555+0000 7ff7eb6f9640 1 -- 172.21.15.40:0/3475506898 >> 172.21.15.40:0/3475506898 conn(0x7ff7e40ce890 msgr2=0x7ff7e41583a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:20.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.555+0000 7ff7eb6f9640 1 -- 172.21.15.40:0/3475506898 shutdown_connections 2023-10-16T08:39:20.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.555+0000 7ff7eb6f9640 1 -- 172.21.15.40:0/3475506898 wait complete. 2023-10-16T08:39:20.576 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:39:20.701 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.695+0000 7f23907f9640 1 Processor -- start 2023-10-16T08:39:20.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.695+0000 7f23907f9640 1 -- start start 2023-10-16T08:39:20.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.695+0000 7f23907f9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2388160510 0x7f23880d2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:20.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.695+0000 7f23907f9640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23880d3300 con 0x7f23880cddc0 2023-10-16T08:39:20.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.695+0000 7f23907f9640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23880d35e0 con 0x7f23880d0310 2023-10-16T08:39:20.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.695+0000 7f23907f9640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23880d38c0 con 0x7f2388160510 2023-10-16T08:39:20.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.695+0000 7f238e595640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f2388160510 0x7f23880d2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:20.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.695+0000 7f238dd94640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f23880cddc0 0x7f23880cfc00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60956/0 (socket says 172.21.15.40:60956) 2023-10-16T08:39:20.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.695+0000 7f238e595640 1 --2- 172.21.15.40:0/4200399392 >> v2:172.21.15.40:3300/0 conn(0x7f2388160510 0x7f23880d2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44934/0 (socket says 172.21.15.40:44934) 2023-10-16T08:39:20.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.695+0000 7f238dd94640 1 -- 172.21.15.40:0/4200399392 learned_addr learned my addr 172.21.15.40:0/4200399392 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:20.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.699+0000 7f238e595640 1 -- 172.21.15.40:0/4200399392 >> v1:172.21.15.86:6789/0 conn(0x7f23880d0310 legacy=0x7f23880b1890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:20.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.699+0000 7f238e595640 1 -- 172.21.15.40:0/4200399392 >> v1:172.21.15.40:6789/0 conn(0x7f23880cddc0 legacy=0x7f23880cfc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:20.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.699+0000 7f238e595640 1 -- 172.21.15.40:0/4200399392 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23880b5f10 con 0x7f2388160510 2023-10-16T08:39:20.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.699+0000 7f238d593640 1 -- 172.21.15.40:0/4200399392 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4129762662 0 0) 0x7f23880d3300 con 0x7f23880cddc0 2023-10-16T08:39:20.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.699+0000 7f238e595640 1 --2- 172.21.15.40:0/4200399392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2388160510 0x7f23880d2c70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f23840097c0 tx=0x7f238401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84358d521ece22dc server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:20.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.699+0000 7f238d593640 1 -- 172.21.15.40:0/4200399392 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23840ab620 con 0x7f2388160510 2023-10-16T08:39:20.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.699+0000 7f238d593640 1 -- 172.21.15.40:0/4200399392 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23840ab780 con 0x7f2388160510 2023-10-16T08:39:20.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.699+0000 7f238d593640 1 -- 172.21.15.40:0/4200399392 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23840abb10 con 0x7f2388160510 2023-10-16T08:39:20.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.699+0000 7f23907f9640 1 -- 172.21.15.40:0/4200399392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2388160510 msgr2=0x7f23880d2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:20.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.699+0000 7f23907f9640 1 --2- 172.21.15.40:0/4200399392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2388160510 0x7f23880d2c70 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f23840097c0 tx=0x7f238401c480 comp rx=0 tx=0).stop 2023-10-16T08:39:20.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.699+0000 7f23907f9640 1 -- 172.21.15.40:0/4200399392 shutdown_connections 2023-10-16T08:39:20.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.699+0000 7f23907f9640 1 --2- 172.21.15.40:0/4200399392 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f2388160510 0x7f23880d2c70 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.699+0000 7f23907f9640 1 -- 172.21.15.40:0/4200399392 >> 172.21.15.40:0/4200399392 conn(0x7f238806a060 msgr2=0x7f238806a470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:20.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.699+0000 7f23907f9640 1 -- 172.21.15.40:0/4200399392 shutdown_connections 2023-10-16T08:39:20.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.699+0000 7f23907f9640 1 -- 172.21.15.40:0/4200399392 wait complete. 2023-10-16T08:39:20.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f23907f9640 1 Processor -- start 2023-10-16T08:39:20.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f23907f9640 1 -- start start 2023-10-16T08:39:20.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f23907f9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23880cddc0 0x7f23880c9280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:20.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f23907f9640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23880d0310 0x7f238815e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:20.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f23907f9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2388160510 0x7f23880d2220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:20.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f23907f9640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238815dc20 con 0x7f23880cddc0 2023-10-16T08:39:20.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f23907f9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238815e4a0 con 0x7f2388160510 2023-10-16T08:39:20.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f23907f9640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23880c97c0 con 0x7f23880d0310 2023-10-16T08:39:20.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f238e595640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23880cddc0 0x7f23880c9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:20.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f238e595640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23880cddc0 0x7f23880c9280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44946/0 (socket says 172.21.15.40:44946) 2023-10-16T08:39:20.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f238e595640 1 -- 172.21.15.40:0/1874147221 learned_addr learned my addr 172.21.15.40:0/1874147221 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:20.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f238e595640 1 -- 172.21.15.40:0/1874147221 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23880d0310 msgr2=0x7f238815e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:20.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f238dd94640 1 --2- 172.21.15.40:0/1874147221 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23880d0310 0x7f238815e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:20.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f238ed96640 1 --2- 172.21.15.40:0/1874147221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2388160510 0x7f23880d2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:20.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f238e595640 1 --2- 172.21.15.40:0/1874147221 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23880d0310 0x7f238815e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f238e595640 1 -- 172.21.15.40:0/1874147221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2388160510 msgr2=0x7f23880d2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:20.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f238e595640 1 --2- 172.21.15.40:0/1874147221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2388160510 0x7f23880d2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f238e595640 1 -- 172.21.15.40:0/1874147221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2384009420 con 0x7f23880cddc0 2023-10-16T08:39:20.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f238ed96640 1 --2- 172.21.15.40:0/1874147221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2388160510 0x7f23880d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:20.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f238dd94640 1 --2- 172.21.15.40:0/1874147221 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23880d0310 0x7f238815e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:20.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f238e595640 1 --2- 172.21.15.40:0/1874147221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23880cddc0 0x7f23880c9280 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2384019090 tx=0x7f238401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:20.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f237effd640 1 -- 172.21.15.40:0/1874147221 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23840bf410 con 0x7f23880cddc0 2023-10-16T08:39:20.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f237effd640 1 -- 172.21.15.40:0/1874147221 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23840bf570 con 0x7f23880cddc0 2023-10-16T08:39:20.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f23907f9640 1 -- 172.21.15.40:0/1874147221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f238814dd80 con 0x7f23880cddc0 2023-10-16T08:39:20.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f237effd640 1 -- 172.21.15.40:0/1874147221 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23840bf920 con 0x7f23880cddc0 2023-10-16T08:39:20.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f23907f9640 1 -- 172.21.15.40:0/1874147221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f238815de50 con 0x7f23880cddc0 2023-10-16T08:39:20.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f23907f9640 1 -- 172.21.15.40:0/1874147221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f238804e9c0 con 0x7f23880cddc0 2023-10-16T08:39:20.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.703+0000 7f237effd640 1 -- 172.21.15.40:0/1874147221 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f23840bfa80 con 0x7f23880cddc0 2023-10-16T08:39:20.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.707+0000 7f237effd640 1 --2- 172.21.15.40:0/1874147221 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f236c0664a0 0x7f236c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:20.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.707+0000 7f237effd640 1 -- 172.21.15.40:0/1874147221 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f238411e4e0 con 0x7f23880cddc0 2023-10-16T08:39:20.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.707+0000 7f238dd94640 1 --2- 172.21.15.40:0/1874147221 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f236c0664a0 0x7f236c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:20.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.707+0000 7f238dd94640 1 --2- 172.21.15.40:0/1874147221 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f236c0664a0 0x7f236c068960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f238814e910 tx=0x7f2378009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:20.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.711+0000 7f237effd640 1 -- 172.21.15.40:0/1874147221 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f23840eba80 con 0x7f23880cddc0 2023-10-16T08:39:20.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.871+0000 7f23907f9640 1 -- 172.21.15.40:0/1874147221 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f23880c9c20 con 0x7f23880cddc0 2023-10-16T08:39:20.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.871+0000 7f237effd640 1 -- 172.21.15.40:0/1874147221 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f23840ef0a0 con 0x7f23880cddc0 2023-10-16T08:39:20.879 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-16T08:39:20.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.875+0000 7f23907f9640 1 -- 172.21.15.40:0/1874147221 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f236c0664a0 msgr2=0x7f236c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:20.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.875+0000 7f23907f9640 1 --2- 172.21.15.40:0/1874147221 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f236c0664a0 0x7f236c068960 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f238814e910 tx=0x7f2378009040 comp rx=0 tx=0).stop 2023-10-16T08:39:20.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.875+0000 7f23907f9640 1 -- 172.21.15.40:0/1874147221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23880cddc0 msgr2=0x7f23880c9280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:20.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.875+0000 7f23907f9640 1 --2- 172.21.15.40:0/1874147221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23880cddc0 0x7f23880c9280 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f2384019090 tx=0x7f238401acc0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.875+0000 7f23907f9640 1 -- 172.21.15.40:0/1874147221 shutdown_connections 2023-10-16T08:39:20.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.875+0000 7f23907f9640 1 --2- 172.21.15.40:0/1874147221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2388160510 0x7f23880d2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.875+0000 7f23907f9640 1 --2- 172.21.15.40:0/1874147221 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23880d0310 0x7f238815e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.875+0000 7f23907f9640 1 --2- 172.21.15.40:0/1874147221 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f236c0664a0 0x7f236c068960 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.875+0000 7f23907f9640 1 --2- 172.21.15.40:0/1874147221 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23880cddc0 0x7f23880c9280 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:20.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.875+0000 7f23907f9640 1 -- 172.21.15.40:0/1874147221 >> 172.21.15.40:0/1874147221 conn(0x7f238806a060 msgr2=0x7f238815f6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:20.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.875+0000 7f23907f9640 1 -- 172.21.15.40:0/1874147221 shutdown_connections 2023-10-16T08:39:20.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:20.875+0000 7f23907f9640 1 -- 172.21.15.40:0/1874147221 wait complete. 2023-10-16T08:39:20.896 DEBUG:teuthology.orchestra.run.smithi040:> set -ex 2023-10-16T08:39:20.896 DEBUG:teuthology.orchestra.run.smithi040:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:39:20.902 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:39:21.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.067+0000 7fd5deedd640 1 Processor -- start 2023-10-16T08:39:21.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.067+0000 7fd5deedd640 1 -- start start 2023-10-16T08:39:21.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.071+0000 7fd5deedd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd5d815cce0 0x7fd5d80b68d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.071+0000 7fd5deedd640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5d80b6e10 0x7fd5d80cd960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.071+0000 7fd5deedd640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd5d80cdea0 0x7fd5d80b4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.071+0000 7fd5deedd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5d80ce330 con 0x7fd5d815cce0 2023-10-16T08:39:21.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.071+0000 7fd5deedd640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5d80b7f80 con 0x7fd5d80cdea0 2023-10-16T08:39:21.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.071+0000 7fd5deedd640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d80b7370 con 0x7fd5d80b6e10 2023-10-16T08:39:21.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.071+0000 7fd5dcc79640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd5d815cce0 0x7fd5d80b68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.071+0000 7fd5dcc79640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd5d815cce0 0x7fd5d80b68d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44962/0 (socket says 172.21.15.40:44962) 2023-10-16T08:39:21.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.071+0000 7fd5dcc79640 1 -- 172.21.15.40:0/2106815161 learned_addr learned my addr 172.21.15.40:0/2106815161 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:21.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.071+0000 7fd5dd47a640 1 --2- 172.21.15.40:0/2106815161 >> v2:172.21.15.86:3300/0 conn(0x7fd5d80cdea0 0x7fd5d80b4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.071+0000 7fd5dcc79640 1 -- 172.21.15.40:0/2106815161 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5d80b6e10 msgr2=0x7fd5d80cd960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:21.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.071+0000 7fd5dcc79640 1 --2- 172.21.15.40:0/2106815161 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5d80b6e10 0x7fd5d80cd960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.071+0000 7fd5dcc79640 1 -- 172.21.15.40:0/2106815161 >> v2:172.21.15.86:3300/0 conn(0x7fd5d80cdea0 msgr2=0x7fd5d80b4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:21.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.071+0000 7fd5dcc79640 1 --2- 172.21.15.40:0/2106815161 >> v2:172.21.15.86:3300/0 conn(0x7fd5d80cdea0 0x7fd5d80b4710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.071+0000 7fd5dcc79640 1 -- 172.21.15.40:0/2106815161 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d80b7650 con 0x7fd5d815cce0 2023-10-16T08:39:21.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.071+0000 7fd5dcc79640 1 --2- 172.21.15.40:0/2106815161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d815cce0 0x7fd5d80b68d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009870 tx=0x7fd5d001c240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb2431539db81207 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:21.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.071+0000 7fd5d77fe640 1 -- 172.21.15.40:0/2106815161 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d00ab600 con 0x7fd5d815cce0 2023-10-16T08:39:21.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.071+0000 7fd5d77fe640 1 -- 172.21.15.40:0/2106815161 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d00a7050 con 0x7fd5d815cce0 2023-10-16T08:39:21.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.071+0000 7fd5d77fe640 1 -- 172.21.15.40:0/2106815161 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d00ab980 con 0x7fd5d815cce0 2023-10-16T08:39:21.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5deedd640 1 -- 172.21.15.40:0/2106815161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d815cce0 msgr2=0x7fd5d80b68d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:21.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5deedd640 1 --2- 172.21.15.40:0/2106815161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d815cce0 0x7fd5d80b68d0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0009870 tx=0x7fd5d001c240 comp rx=0 tx=0).stop 2023-10-16T08:39:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5deedd640 1 -- 172.21.15.40:0/2106815161 shutdown_connections 2023-10-16T08:39:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5deedd640 1 --2- 172.21.15.40:0/2106815161 >> v2:172.21.15.86:3300/0 conn(0x7fd5d80cdea0 0x7fd5d80b4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5deedd640 1 --2- 172.21.15.40:0/2106815161 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5d80b6e10 0x7fd5d80cd960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5deedd640 1 --2- 172.21.15.40:0/2106815161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d815cce0 0x7fd5d80b68d0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5deedd640 1 -- 172.21.15.40:0/2106815161 >> 172.21.15.40:0/2106815161 conn(0x7fd5d806a290 msgr2=0x7fd5d806a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:21.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5deedd640 1 -- 172.21.15.40:0/2106815161 shutdown_connections 2023-10-16T08:39:21.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5deedd640 1 -- 172.21.15.40:0/2106815161 wait complete. 2023-10-16T08:39:21.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5deedd640 1 Processor -- start 2023-10-16T08:39:21.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5deedd640 1 -- start start 2023-10-16T08:39:21.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5deedd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5d80b6e10 0x7fd5d8147c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5deedd640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5d80cdea0 0x7fd5d8153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5deedd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d815cce0 0x7fd5d8153890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5deedd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d815e1d0 con 0x7fd5d815cce0 2023-10-16T08:39:21.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5deedd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d815e4b0 con 0x7fd5d80b6e10 2023-10-16T08:39:21.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5deedd640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5d815e790 con 0x7fd5d80cdea0 2023-10-16T08:39:21.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5dd47a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d815cce0 0x7fd5d8153890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5dd47a640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d815cce0 0x7fd5d8153890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44974/0 (socket says 172.21.15.40:44974) 2023-10-16T08:39:21.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5dd47a640 1 -- 172.21.15.40:0/718706642 learned_addr learned my addr 172.21.15.40:0/718706642 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:21.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5dcc79640 1 --2- 172.21.15.40:0/718706642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5d80b6e10 0x7fd5d8147c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5d7fff640 1 --2- 172.21.15.40:0/718706642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5d80cdea0 0x7fd5d8153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5dd47a640 1 -- 172.21.15.40:0/718706642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5d80cdea0 msgr2=0x7fd5d8153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5dd47a640 1 --2- 172.21.15.40:0/718706642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5d80cdea0 0x7fd5d8153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5dd47a640 1 -- 172.21.15.40:0/718706642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5d80b6e10 msgr2=0x7fd5d8147c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5dd47a640 1 --2- 172.21.15.40:0/718706642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5d80b6e10 0x7fd5d8147c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5dd47a640 1 -- 172.21.15.40:0/718706642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d00094d0 con 0x7fd5d815cce0 2023-10-16T08:39:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5dd47a640 1 --2- 172.21.15.40:0/718706642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d815cce0 0x7fd5d8153890 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800d800 tx=0x7fd5c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5d57fa640 1 -- 172.21.15.40:0/718706642 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c800b6c0 con 0x7fd5d815cce0 2023-10-16T08:39:21.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5d57fa640 1 -- 172.21.15.40:0/718706642 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5c800b820 con 0x7fd5d815cce0 2023-10-16T08:39:21.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5deedd640 1 -- 172.21.15.40:0/718706642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5d815ea70 con 0x7fd5d815cce0 2023-10-16T08:39:21.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5d57fa640 1 -- 172.21.15.40:0/718706642 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5c800bbd0 con 0x7fd5d815cce0 2023-10-16T08:39:21.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.075+0000 7fd5deedd640 1 -- 172.21.15.40:0/718706642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5d81495f0 con 0x7fd5d815cce0 2023-10-16T08:39:21.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.079+0000 7fd5deedd640 1 -- 172.21.15.40:0/718706642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5d804e9c0 con 0x7fd5d815cce0 2023-10-16T08:39:21.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.079+0000 7fd5d57fa640 1 -- 172.21.15.40:0/718706642 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd5c800bd30 con 0x7fd5d815cce0 2023-10-16T08:39:21.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.079+0000 7fd5d57fa640 1 --2- 172.21.15.40:0/718706642 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd5b80664a0 0x7fd5b8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.079+0000 7fd5d57fa640 1 -- 172.21.15.40:0/718706642 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd5c8084460 con 0x7fd5d815cce0 2023-10-16T08:39:21.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.079+0000 7fd5dcc79640 1 --2- 172.21.15.40:0/718706642 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd5b80664a0 0x7fd5b8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.079+0000 7fd5dcc79640 1 --2- 172.21.15.40:0/718706642 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd5b80664a0 0x7fd5b8068960 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00ad000 tx=0x7fd5d0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:21.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.083+0000 7fd5d57fa640 1 -- 172.21.15.40:0/718706642 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fd5c8051850 con 0x7fd5d815cce0 2023-10-16T08:39:21.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.243+0000 7fd5deedd640 1 -- 172.21.15.40:0/718706642 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd5d8156f30 con 0x7fd5d815cce0 2023-10-16T08:39:21.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.243+0000 7fd5d57fa640 1 -- 172.21.15.40:0/718706642 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16445 (secure 0 0 0) 0x7fd5c8054e70 con 0x7fd5d815cce0 2023-10-16T08:39:21.251 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:39:21.251 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:46.611098+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:39:21.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.247+0000 7fd5deedd640 1 -- 172.21.15.40:0/718706642 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd5b80664a0 msgr2=0x7fd5b8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:21.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.247+0000 7fd5deedd640 1 --2- 172.21.15.40:0/718706642 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd5b80664a0 0x7fd5b8068960 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fd5d00ad000 tx=0x7fd5d0008040 comp rx=0 tx=0).stop 2023-10-16T08:39:21.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.247+0000 7fd5deedd640 1 -- 172.21.15.40:0/718706642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d815cce0 msgr2=0x7fd5d8153890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:21.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.247+0000 7fd5deedd640 1 --2- 172.21.15.40:0/718706642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d815cce0 0x7fd5d8153890 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fd5c800d800 tx=0x7fd5c800dcc0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.247+0000 7fd5deedd640 1 -- 172.21.15.40:0/718706642 shutdown_connections 2023-10-16T08:39:21.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.247+0000 7fd5deedd640 1 --2- 172.21.15.40:0/718706642 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd5b80664a0 0x7fd5b8068960 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.247+0000 7fd5deedd640 1 --2- 172.21.15.40:0/718706642 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd5d815cce0 0x7fd5d8153890 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.247+0000 7fd5deedd640 1 --2- 172.21.15.40:0/718706642 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd5d80cdea0 0x7fd5d8153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.247+0000 7fd5deedd640 1 --2- 172.21.15.40:0/718706642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd5d80b6e10 0x7fd5d8147c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.247+0000 7fd5deedd640 1 -- 172.21.15.40:0/718706642 >> 172.21.15.40:0/718706642 conn(0x7fd5d806a290 msgr2=0x7fd5d80d1a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:21.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.247+0000 7fd5deedd640 1 -- 172.21.15.40:0/718706642 shutdown_connections 2023-10-16T08:39:21.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.247+0000 7fd5deedd640 1 -- 172.21.15.40:0/718706642 wait complete. 2023-10-16T08:39:21.271 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:38:33.014668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:38:43.055215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:38:43.971666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:39:21.272 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:39:21.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa437691640 1 Processor -- start 2023-10-16T08:39:21.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa437691640 1 -- start start 2023-10-16T08:39:21.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa437691640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa4300adad0 0x7fa4300b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa437691640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4300c98b0 con 0x7fa4300cae10 2023-10-16T08:39:21.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa437691640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4300b47c0 con 0x7fa4300adeb0 2023-10-16T08:39:21.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa437691640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4300c8f80 con 0x7fa4300adad0 2023-10-16T08:39:21.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa435c2e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa4300cae10 0x7fa43015a500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60962/0 (socket says 172.21.15.40:60962) 2023-10-16T08:39:21.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa435c2e640 1 -- 172.21.15.40:0/3914447415 learned_addr learned my addr 172.21.15.40:0/3914447415 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:21.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa43542d640 1 --2- 172.21.15.40:0/3914447415 >> v2:172.21.15.86:3300/0 conn(0x7fa4300adad0 0x7fa4300b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa427fff640 1 -- 172.21.15.40:0/3914447415 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 495564651 0 0) 0x7fa4300c98b0 con 0x7fa4300cae10 2023-10-16T08:39:21.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa427fff640 1 -- 172.21.15.40:0/3914447415 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa41c003530 con 0x7fa4300cae10 2023-10-16T08:39:21.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa427fff640 1 -- 172.21.15.40:0/3914447415 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2888755110 0 0) 0x7fa41c003530 con 0x7fa4300cae10 2023-10-16T08:39:21.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa427fff640 1 -- 172.21.15.40:0/3914447415 >> v2:172.21.15.86:3300/0 conn(0x7fa4300adad0 msgr2=0x7fa4300b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:21.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa427fff640 1 --2- 172.21.15.40:0/3914447415 >> v2:172.21.15.86:3300/0 conn(0x7fa4300adad0 0x7fa4300b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa427fff640 1 -- 172.21.15.40:0/3914447415 >> v1:172.21.15.86:6789/0 conn(0x7fa4300adeb0 legacy=0x7fa4300ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa427fff640 1 -- 172.21.15.40:0/3914447415 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4300c9260 con 0x7fa4300cae10 2023-10-16T08:39:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa427fff640 1 -- 172.21.15.40:0/3914447415 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7fa428001a00 con 0x7fa4300cae10 2023-10-16T08:39:21.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa427fff640 1 -- 172.21.15.40:0/3914447415 >> v1:172.21.15.40:6789/0 conn(0x7fa4300cae10 legacy=0x7fa43015a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:21.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa427fff640 1 --2- 172.21.15.40:0/3914447415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41c0039f0 0x7fa41c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa427fff640 1 --2- 172.21.15.40:0/3914447415 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41c0047d0 0x7fa41c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa427fff640 1 --2- 172.21.15.40:0/3914447415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41c005190 0x7fa41c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa427fff640 1 -- 172.21.15.40:0/3914447415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4300c8f80 con 0x7fa41c005190 2023-10-16T08:39:21.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa427fff640 1 -- 172.21.15.40:0/3914447415 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c00c0c0 con 0x7fa41c0039f0 2023-10-16T08:39:21.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa427fff640 1 -- 172.21.15.40:0/3914447415 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa41c00c280 con 0x7fa41c0047d0 2023-10-16T08:39:21.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa435c2e640 1 --2- 172.21.15.40:0/3914447415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41c005190 0x7fa41c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa43542d640 1 --2- 172.21.15.40:0/3914447415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41c0039f0 0x7fa41c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa434c2c640 1 --2- 172.21.15.40:0/3914447415 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41c0047d0 0x7fa41c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa435c2e640 1 -- 172.21.15.40:0/3914447415 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41c0047d0 msgr2=0x7fa41c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:21.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa435c2e640 1 --2- 172.21.15.40:0/3914447415 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41c0047d0 0x7fa41c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa435c2e640 1 -- 172.21.15.40:0/3914447415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41c0039f0 msgr2=0x7fa41c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:21.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa435c2e640 1 --2- 172.21.15.40:0/3914447415 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa41c0039f0 0x7fa41c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.391+0000 7fa435c2e640 1 -- 172.21.15.40:0/3914447415 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa41c003530 con 0x7fa41c005190 2023-10-16T08:39:21.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.395+0000 7fa435c2e640 1 -- 172.21.15.40:0/3914447415 reap_dead start 2023-10-16T08:39:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.395+0000 7fa435c2e640 1 --2- 172.21.15.40:0/3914447415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41c005190 0x7fa41c00b990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa42800a830 tx=0x7fa42801d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a078779e0bc721 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.395+0000 7fa427fff640 1 -- 172.21.15.40:0/3914447415 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4280072d0 con 0x7fa41c005190 2023-10-16T08:39:21.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.395+0000 7fa427fff640 1 -- 172.21.15.40:0/3914447415 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4280a8070 con 0x7fa41c005190 2023-10-16T08:39:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.395+0000 7fa427fff640 1 -- 172.21.15.40:0/3914447415 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa428007690 con 0x7fa41c005190 2023-10-16T08:39:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.395+0000 7fa437691640 1 -- 172.21.15.40:0/3914447415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41c005190 msgr2=0x7fa41c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:21.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.395+0000 7fa437691640 1 --2- 172.21.15.40:0/3914447415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41c005190 0x7fa41c00b990 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa42800a830 tx=0x7fa42801d340 comp rx=0 tx=0).stop 2023-10-16T08:39:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.395+0000 7fa437691640 1 -- 172.21.15.40:0/3914447415 shutdown_connections 2023-10-16T08:39:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.395+0000 7fa437691640 1 --2- 172.21.15.40:0/3914447415 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa41c005190 0x7fa41c00b990 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.395+0000 7fa437691640 1 -- 172.21.15.40:0/3914447415 >> 172.21.15.40:0/3914447415 conn(0x7fa4300ce890 msgr2=0x7fa4300ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:21.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.395+0000 7fa437691640 1 -- 172.21.15.40:0/3914447415 shutdown_connections 2023-10-16T08:39:21.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.395+0000 7fa437691640 1 -- 172.21.15.40:0/3914447415 wait complete. 2023-10-16T08:39:21.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.395+0000 7fa437691640 1 Processor -- start 2023-10-16T08:39:21.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.395+0000 7fa437691640 1 -- start start 2023-10-16T08:39:21.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa437691640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41c005190 0x7fa4301f5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa437691640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4301567c0 0x7fa430151fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa437691640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa430158190 0x7fa430158610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa437691640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa430153fe0 con 0x7fa4301567c0 2023-10-16T08:39:21.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa437691640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4301f61a0 con 0x7fa430158190 2023-10-16T08:39:21.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa437691640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4301f6480 con 0x7fa41c005190 2023-10-16T08:39:21.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa434c2c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4301567c0 0x7fa430151fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa434c2c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4301567c0 0x7fa430151fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:44992/0 (socket says 172.21.15.40:44992) 2023-10-16T08:39:21.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa434c2c640 1 -- 172.21.15.40:0/2061098919 learned_addr learned my addr 172.21.15.40:0/2061098919 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:21.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa435c2e640 1 --2- 172.21.15.40:0/2061098919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa430158190 0x7fa430158610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa43542d640 1 --2- 172.21.15.40:0/2061098919 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41c005190 0x7fa4301f5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa434c2c640 1 -- 172.21.15.40:0/2061098919 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41c005190 msgr2=0x7fa4301f5950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:21.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa434c2c640 1 --2- 172.21.15.40:0/2061098919 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41c005190 0x7fa4301f5950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa434c2c640 1 -- 172.21.15.40:0/2061098919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa430158190 msgr2=0x7fa430158610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:21.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa434c2c640 1 --2- 172.21.15.40:0/2061098919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa430158190 0x7fa430158610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa434c2c640 1 -- 172.21.15.40:0/2061098919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4301f6760 con 0x7fa4301567c0 2023-10-16T08:39:21.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa434c2c640 1 --2- 172.21.15.40:0/2061098919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4301567c0 0x7fa430151fe0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa42000e410 tx=0x7fa42000e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:21.418 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa425ffb640 1 -- 172.21.15.40:0/2061098919 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4200052e0 con 0x7fa4301567c0 2023-10-16T08:39:21.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa425ffb640 1 -- 172.21.15.40:0/2061098919 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa420005440 con 0x7fa4301567c0 2023-10-16T08:39:21.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa437691640 1 -- 172.21.15.40:0/2061098919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa430154280 con 0x7fa4301567c0 2023-10-16T08:39:21.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa425ffb640 1 -- 172.21.15.40:0/2061098919 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa420003820 con 0x7fa4301567c0 2023-10-16T08:39:21.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa437691640 1 -- 172.21.15.40:0/2061098919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa430152520 con 0x7fa4301567c0 2023-10-16T08:39:21.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa437691640 1 -- 172.21.15.40:0/2061098919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa430067a20 con 0x7fa4301567c0 2023-10-16T08:39:21.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa425ffb640 1 -- 172.21.15.40:0/2061098919 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa420003980 con 0x7fa4301567c0 2023-10-16T08:39:21.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa425ffb640 1 --2- 172.21.15.40:0/2061098919 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa4140664d0 0x7fa414068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa425ffb640 1 -- 172.21.15.40:0/2061098919 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa420083590 con 0x7fa4301567c0 2023-10-16T08:39:21.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.399+0000 7fa43542d640 1 --2- 172.21.15.40:0/2061098919 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa4140664d0 0x7fa414068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.403+0000 7fa43542d640 1 --2- 172.21.15.40:0/2061098919 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa4140664d0 0x7fa414068990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa4300b46c0 tx=0x7fa42c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:21.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.403+0000 7fa425ffb640 1 -- 172.21.15.40:0/2061098919 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fa420050b50 con 0x7fa4301567c0 2023-10-16T08:39:21.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.567+0000 7fa437691640 1 -- 172.21.15.40:0/2061098919 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa43004e9c0 con 0x7fa4301567c0 2023-10-16T08:39:21.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.567+0000 7fa425ffb640 1 -- 172.21.15.40:0/2061098919 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa420019020 con 0x7fa4301567c0 2023-10-16T08:39:21.572 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-16T08:39:21.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.567+0000 7fa437691640 1 -- 172.21.15.40:0/2061098919 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa4140664d0 msgr2=0x7fa414068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:21.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.567+0000 7fa437691640 1 --2- 172.21.15.40:0/2061098919 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa4140664d0 0x7fa414068990 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fa4300b46c0 tx=0x7fa42c008040 comp rx=0 tx=0).stop 2023-10-16T08:39:21.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.567+0000 7fa437691640 1 -- 172.21.15.40:0/2061098919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4301567c0 msgr2=0x7fa430151fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:21.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.567+0000 7fa437691640 1 --2- 172.21.15.40:0/2061098919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4301567c0 0x7fa430151fe0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7fa42000e410 tx=0x7fa42000e8d0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.567+0000 7fa437691640 1 -- 172.21.15.40:0/2061098919 shutdown_connections 2023-10-16T08:39:21.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.567+0000 7fa437691640 1 --2- 172.21.15.40:0/2061098919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa430158190 0x7fa430158610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.567+0000 7fa437691640 1 --2- 172.21.15.40:0/2061098919 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa4140664d0 0x7fa414068990 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.567+0000 7fa437691640 1 --2- 172.21.15.40:0/2061098919 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa4301567c0 0x7fa430151fe0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.567+0000 7fa437691640 1 --2- 172.21.15.40:0/2061098919 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa41c005190 0x7fa4301f5950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.567+0000 7fa437691640 1 -- 172.21.15.40:0/2061098919 >> 172.21.15.40:0/2061098919 conn(0x7fa4300ce890 msgr2=0x7fa43006b3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:21.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.567+0000 7fa437691640 1 -- 172.21.15.40:0/2061098919 shutdown_connections 2023-10-16T08:39:21.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.567+0000 7fa437691640 1 -- 172.21.15.40:0/2061098919 wait complete. 2023-10-16T08:39:21.591 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:39:21.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a1bdb2640 1 Processor -- start 2023-10-16T08:39:21.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a1bdb2640 1 -- start start 2023-10-16T08:39:21.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a1bdb2640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9a140adeb0 0x7f9a140b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a1bdb2640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a140c98b0 con 0x7f9a140adad0 2023-10-16T08:39:21.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a1bdb2640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a140b47c0 con 0x7f9a140cae10 2023-10-16T08:39:21.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a1bdb2640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a140c8f80 con 0x7f9a140adeb0 2023-10-16T08:39:21.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a19b4e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f9a140adad0 0x7f9a140ca700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60968/0 (socket says 172.21.15.40:60968) 2023-10-16T08:39:21.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a19b4e640 1 -- 172.21.15.40:0/3446103959 learned_addr learned my addr 172.21.15.40:0/3446103959 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:21.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a1a34f640 1 --2- 172.21.15.40:0/3446103959 >> v2:172.21.15.86:3300/0 conn(0x7f9a140adeb0 0x7f9a140b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a18b4c640 1 -- 172.21.15.40:0/3446103959 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4038485518 0 0) 0x7f9a140c98b0 con 0x7f9a140adad0 2023-10-16T08:39:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a18b4c640 1 -- 172.21.15.40:0/3446103959 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a00003530 con 0x7f9a140adad0 2023-10-16T08:39:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a18b4c640 1 -- 172.21.15.40:0/3446103959 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 306957019 0 0) 0x7f9a140b47c0 con 0x7f9a140cae10 2023-10-16T08:39:21.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a18b4c640 1 -- 172.21.15.40:0/3446103959 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9a140c98b0 con 0x7f9a140cae10 2023-10-16T08:39:21.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a18b4c640 1 -- 172.21.15.40:0/3446103959 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 184922103 0 0) 0x7f9a00003530 con 0x7f9a140adad0 2023-10-16T08:39:21.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a18b4c640 1 -- 172.21.15.40:0/3446103959 >> v2:172.21.15.86:3300/0 conn(0x7f9a140adeb0 msgr2=0x7f9a140b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:21.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a18b4c640 1 --2- 172.21.15.40:0/3446103959 >> v2:172.21.15.86:3300/0 conn(0x7f9a140adeb0 0x7f9a140b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a18b4c640 1 -- 172.21.15.40:0/3446103959 >> v1:172.21.15.86:6789/0 conn(0x7f9a140cae10 legacy=0x7f9a1415a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:21.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a18b4c640 1 -- 172.21.15.40:0/3446103959 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a140c9260 con 0x7f9a140adad0 2023-10-16T08:39:21.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a18b4c640 1 -- 172.21.15.40:0/3446103959 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f9a10001680 con 0x7f9a140adad0 2023-10-16T08:39:21.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a18b4c640 1 -- 172.21.15.40:0/3446103959 >> v1:172.21.15.40:6789/0 conn(0x7f9a140adad0 legacy=0x7f9a140ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:21.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a18b4c640 1 --2- 172.21.15.40:0/3446103959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a00003b80 0x7f9a00003f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a18b4c640 1 --2- 172.21.15.40:0/3446103959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a000047d0 0x7f9a00004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a18b4c640 1 --2- 172.21.15.40:0/3446103959 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9a00005190 0x7f9a0000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a18b4c640 1 -- 172.21.15.40:0/3446103959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140c8f80 con 0x7f9a00003b80 2023-10-16T08:39:21.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a18b4c640 1 -- 172.21.15.40:0/3446103959 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0000c090 con 0x7f9a000047d0 2023-10-16T08:39:21.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a18b4c640 1 -- 172.21.15.40:0/3446103959 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0000c250 con 0x7f9a00005190 2023-10-16T08:39:21.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a19b4e640 1 --2- 172.21.15.40:0/3446103959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a00003b80 0x7f9a00003f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a1934d640 1 --2- 172.21.15.40:0/3446103959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a000047d0 0x7f9a00004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a1a34f640 1 --2- 172.21.15.40:0/3446103959 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9a00005190 0x7f9a0000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a19b4e640 1 -- 172.21.15.40:0/3446103959 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9a00005190 msgr2=0x7f9a0000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:21.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a19b4e640 1 --2- 172.21.15.40:0/3446103959 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9a00005190 0x7f9a0000b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a19b4e640 1 -- 172.21.15.40:0/3446103959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a000047d0 msgr2=0x7f9a00004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:21.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a19b4e640 1 --2- 172.21.15.40:0/3446103959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a000047d0 0x7f9a00004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a19b4e640 1 -- 172.21.15.40:0/3446103959 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a00003530 con 0x7f9a00003b80 2023-10-16T08:39:21.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.719+0000 7f9a1a34f640 1 -- 172.21.15.40:0/3446103959 reap_dead start 2023-10-16T08:39:21.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.723+0000 7f9a19b4e640 1 --2- 172.21.15.40:0/3446103959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a00003b80 0x7f9a00003f80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000c160 tx=0x7f9a1001cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96cca09fcd5ab47a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.723+0000 7f9a18b4c640 1 -- 172.21.15.40:0/3446103959 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a100058e0 con 0x7f9a00003b80 2023-10-16T08:39:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.723+0000 7f9a18b4c640 1 -- 172.21.15.40:0/3446103959 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a10005a60 con 0x7f9a00003b80 2023-10-16T08:39:21.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.723+0000 7f9a18b4c640 1 -- 172.21.15.40:0/3446103959 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a10005e50 con 0x7f9a00003b80 2023-10-16T08:39:21.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.723+0000 7f9a1bdb2640 1 -- 172.21.15.40:0/3446103959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a00003b80 msgr2=0x7f9a00003f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:21.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.723+0000 7f9a1bdb2640 1 --2- 172.21.15.40:0/3446103959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a00003b80 0x7f9a00003f80 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9a1000c160 tx=0x7f9a1001cb50 comp rx=0 tx=0).stop 2023-10-16T08:39:21.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.723+0000 7f9a1bdb2640 1 -- 172.21.15.40:0/3446103959 shutdown_connections 2023-10-16T08:39:21.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.723+0000 7f9a1bdb2640 1 --2- 172.21.15.40:0/3446103959 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a00003b80 0x7f9a00003f80 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.723+0000 7f9a1bdb2640 1 -- 172.21.15.40:0/3446103959 >> 172.21.15.40:0/3446103959 conn(0x7f9a140ce890 msgr2=0x7f9a140ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:21.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.723+0000 7f9a1bdb2640 1 -- 172.21.15.40:0/3446103959 shutdown_connections 2023-10-16T08:39:21.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.723+0000 7f9a1bdb2640 1 -- 172.21.15.40:0/3446103959 wait complete. 2023-10-16T08:39:21.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1bdb2640 1 Processor -- start 2023-10-16T08:39:21.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1bdb2640 1 -- start start 2023-10-16T08:39:21.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1bdb2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9a00003b80 0x7f9a140c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1bdb2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a140c8520 0x7f9a140b2940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1bdb2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1414b8d0 0x7f9a1414bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1bdb2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140c8d90 con 0x7f9a1414b8d0 2023-10-16T08:39:21.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1bdb2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a1414c290 con 0x7f9a140c8520 2023-10-16T08:39:21.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1bdb2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a140c3960 con 0x7f9a00003b80 2023-10-16T08:39:21.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1a34f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1414b8d0 0x7f9a1414bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1a34f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1414b8d0 0x7f9a1414bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45002/0 (socket says 172.21.15.40:45002) 2023-10-16T08:39:21.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1a34f640 1 -- 172.21.15.40:0/12959624 learned_addr learned my addr 172.21.15.40:0/12959624 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:21.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1934d640 1 --2- 172.21.15.40:0/12959624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a140c8520 0x7f9a140b2940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a19b4e640 1 --2- 172.21.15.40:0/12959624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9a00003b80 0x7f9a140c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1a34f640 1 -- 172.21.15.40:0/12959624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9a00003b80 msgr2=0x7f9a140c5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:21.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1a34f640 1 --2- 172.21.15.40:0/12959624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9a00003b80 0x7f9a140c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1a34f640 1 -- 172.21.15.40:0/12959624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a140c8520 msgr2=0x7f9a140b2940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:21.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1a34f640 1 --2- 172.21.15.40:0/12959624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a140c8520 0x7f9a140b2940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1a34f640 1 -- 172.21.15.40:0/12959624 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a140c3c40 con 0x7f9a1414b8d0 2023-10-16T08:39:21.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1934d640 1 --2- 172.21.15.40:0/12959624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a140c8520 0x7f9a140b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:21.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1a34f640 1 --2- 172.21.15.40:0/12959624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1414b8d0 0x7f9a1414bd50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9a14070220 tx=0x7f9a0c00f520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:21.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a0a7fc640 1 -- 172.21.15.40:0/12959624 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c01a070 con 0x7f9a1414b8d0 2023-10-16T08:39:21.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a0a7fc640 1 -- 172.21.15.40:0/12959624 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a0c00fdf0 con 0x7f9a1414b8d0 2023-10-16T08:39:21.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a0a7fc640 1 -- 172.21.15.40:0/12959624 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a0c003bc0 con 0x7f9a1414b8d0 2023-10-16T08:39:21.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1bdb2640 1 -- 172.21.15.40:0/12959624 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a140cc500 con 0x7f9a1414b8d0 2023-10-16T08:39:21.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1bdb2640 1 -- 172.21.15.40:0/12959624 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a140c89a0 con 0x7f9a1414b8d0 2023-10-16T08:39:21.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.727+0000 7f9a1bdb2640 1 -- 172.21.15.40:0/12959624 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a14067a20 con 0x7f9a1414b8d0 2023-10-16T08:39:21.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.731+0000 7f9a0a7fc640 1 -- 172.21.15.40:0/12959624 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9a0c016420 con 0x7f9a1414b8d0 2023-10-16T08:39:21.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.731+0000 7f9a0a7fc640 1 --2- 172.21.15.40:0/12959624 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f99f80664d0 0x7f99f8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:21.749 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.731+0000 7f9a0a7fc640 1 -- 172.21.15.40:0/12959624 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9a0c084830 con 0x7f9a1414b8d0 2023-10-16T08:39:21.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.731+0000 7f9a19b4e640 1 --2- 172.21.15.40:0/12959624 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f99f80664d0 0x7f99f8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:21.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.731+0000 7f9a19b4e640 1 --2- 172.21.15.40:0/12959624 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f99f80664d0 0x7f99f8068990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9a100aa000 tx=0x7f9a10001a00 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:21.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.735+0000 7f9a0a7fc640 1 -- 172.21.15.40:0/12959624 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f9a0c051d40 con 0x7f9a1414b8d0 2023-10-16T08:39:21.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.891+0000 7f9a1bdb2640 1 -- 172.21.15.40:0/12959624 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9a140cc6f0 con 0x7f9a1414b8d0 2023-10-16T08:39:21.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.891+0000 7f9a0a7fc640 1 -- 172.21.15.40:0/12959624 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9a0c055360 con 0x7f9a1414b8d0 2023-10-16T08:39:21.899 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-16T08:39:21.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.895+0000 7f9a1bdb2640 1 -- 172.21.15.40:0/12959624 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f99f80664d0 msgr2=0x7f99f8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:21.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.895+0000 7f9a1bdb2640 1 --2- 172.21.15.40:0/12959624 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f99f80664d0 0x7f99f8068990 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9a100aa000 tx=0x7f9a10001a00 comp rx=0 tx=0).stop 2023-10-16T08:39:21.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.895+0000 7f9a1bdb2640 1 -- 172.21.15.40:0/12959624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1414b8d0 msgr2=0x7f9a1414bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:21.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.895+0000 7f9a1bdb2640 1 --2- 172.21.15.40:0/12959624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1414b8d0 0x7f9a1414bd50 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f9a14070220 tx=0x7f9a0c00f520 comp rx=0 tx=0).stop 2023-10-16T08:39:21.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.895+0000 7f9a1bdb2640 1 -- 172.21.15.40:0/12959624 shutdown_connections 2023-10-16T08:39:21.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.895+0000 7f9a1bdb2640 1 --2- 172.21.15.40:0/12959624 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f99f80664d0 0x7f99f8068990 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.895+0000 7f9a1bdb2640 1 --2- 172.21.15.40:0/12959624 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9a1414b8d0 0x7f9a1414bd50 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.895+0000 7f9a1bdb2640 1 --2- 172.21.15.40:0/12959624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a140c8520 0x7f9a140b2940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.895+0000 7f9a1bdb2640 1 --2- 172.21.15.40:0/12959624 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9a00003b80 0x7f9a140c5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:21.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.895+0000 7f9a1bdb2640 1 -- 172.21.15.40:0/12959624 >> 172.21.15.40:0/12959624 conn(0x7f9a140ce890 msgr2=0x7f9a140c25d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:21.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.895+0000 7f9a1bdb2640 1 -- 172.21.15.40:0/12959624 shutdown_connections 2023-10-16T08:39:21.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:21.895+0000 7f9a1bdb2640 1 -- 172.21.15.40:0/12959624 wait complete. 2023-10-16T08:39:21.918 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:39:22.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.043+0000 7f78cf247640 1 Processor -- start 2023-10-16T08:39:22.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.043+0000 7f78cf247640 1 -- start start 2023-10-16T08:39:22.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.043+0000 7f78cf247640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f78c80ce030 0x7f78c8157200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:22.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.043+0000 7f78cf247640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f78c80c9740 0x7f78c8157740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:22.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.043+0000 7f78cf247640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78c806dc40 con 0x7f78c80ce410 2023-10-16T08:39:22.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.043+0000 7f78cf247640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78c806d960 con 0x7f78c80ce030 2023-10-16T08:39:22.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.043+0000 7f78cf247640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78c806fc40 con 0x7f78c80c9740 2023-10-16T08:39:22.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.043+0000 7f78ccfe3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f78c80ce030 0x7f78c8157200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:22.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.043+0000 7f78ccfe3640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f78c80ce030 0x7f78c8157200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45006/0 (socket says 172.21.15.40:45006) 2023-10-16T08:39:22.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.043+0000 7f78ccfe3640 1 -- 172.21.15.40:0/290982723 learned_addr learned my addr 172.21.15.40:0/290982723 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:22.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.043+0000 7f78cd7e4640 1 --1- 172.21.15.40:0/290982723 >> v1:172.21.15.40:6789/0 conn(0x7f78c80ce410 0x7f78c806cbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60970/0 (socket says 172.21.15.40:60970) 2023-10-16T08:39:22.052 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.043+0000 7f78bffff640 1 --2- 172.21.15.40:0/290982723 >> v2:172.21.15.86:3300/0 conn(0x7f78c80c9740 0x7f78c8157740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:22.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.043+0000 7f78ccfe3640 1 -- 172.21.15.40:0/290982723 >> v2:172.21.15.86:3300/0 conn(0x7f78c80c9740 msgr2=0x7f78c8157740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:22.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.043+0000 7f78ccfe3640 1 --2- 172.21.15.40:0/290982723 >> v2:172.21.15.86:3300/0 conn(0x7f78c80c9740 0x7f78c8157740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.053 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.043+0000 7f78ccfe3640 1 -- 172.21.15.40:0/290982723 >> v1:172.21.15.40:6789/0 conn(0x7f78c80ce410 legacy=0x7f78c806cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:22.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.043+0000 7f78ccfe3640 1 -- 172.21.15.40:0/290982723 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c806ff20 con 0x7f78c80ce030 2023-10-16T08:39:22.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.043+0000 7f78bf7fe640 1 -- 172.21.15.40:0/290982723 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1351053775 0 0) 0x7f78c806dc40 con 0x7f78c80ce410 2023-10-16T08:39:22.054 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.043+0000 7f78bffff640 1 --2- 172.21.15.40:0/290982723 >> v2:172.21.15.86:3300/0 conn(0x7f78c80c9740 0x7f78c8157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:22.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.047+0000 7f78ccfe3640 1 --2- 172.21.15.40:0/290982723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c80ce030 0x7f78c8157200 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f78c400bb10 tx=0x7f78c401ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7bca25f83a0875a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:22.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.047+0000 7f78bf7fe640 1 -- 172.21.15.40:0/290982723 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c40a79a0 con 0x7f78c80ce030 2023-10-16T08:39:22.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.047+0000 7f78bf7fe640 1 -- 172.21.15.40:0/290982723 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c40a7b00 con 0x7f78c80ce030 2023-10-16T08:39:22.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.047+0000 7f78bf7fe640 1 -- 172.21.15.40:0/290982723 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c40a7e90 con 0x7f78c80ce030 2023-10-16T08:39:22.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.047+0000 7f78cf247640 1 -- 172.21.15.40:0/290982723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c80ce030 msgr2=0x7f78c8157200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:22.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.047+0000 7f78cf247640 1 --2- 172.21.15.40:0/290982723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c80ce030 0x7f78c8157200 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f78c400bb10 tx=0x7f78c401ebd0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.047+0000 7f78cf247640 1 -- 172.21.15.40:0/290982723 shutdown_connections 2023-10-16T08:39:22.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.047+0000 7f78cf247640 1 --2- 172.21.15.40:0/290982723 >> v2:172.21.15.86:3300/0 conn(0x7f78c80c9740 0x7f78c8157740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.047+0000 7f78cf247640 1 --2- 172.21.15.40:0/290982723 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c80ce030 0x7f78c8157200 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.047+0000 7f78cf247640 1 -- 172.21.15.40:0/290982723 >> 172.21.15.40:0/290982723 conn(0x7f78c80cf0f0 msgr2=0x7f78c80cf500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:22.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.047+0000 7f78cf247640 1 -- 172.21.15.40:0/290982723 shutdown_connections 2023-10-16T08:39:22.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.047+0000 7f78cf247640 1 -- 172.21.15.40:0/290982723 wait complete. 2023-10-16T08:39:22.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78cf247640 1 Processor -- start 2023-10-16T08:39:22.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78cf247640 1 -- start start 2023-10-16T08:39:22.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78cf247640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c80c9740 0x7f78c8155b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:22.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78cf247640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f78c80ce030 0x7f78c814c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:22.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78cf247640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78c80ce410 0x7f78c80c3860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:22.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78cf247640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c8156080 con 0x7f78c80c9740 2023-10-16T08:39:22.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78cf247640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78c814f850 con 0x7f78c80ce410 2023-10-16T08:39:22.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78cf247640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78c80c7ce0 con 0x7f78c80ce030 2023-10-16T08:39:22.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78ccfe3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c80c9740 0x7f78c8155b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:22.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78ccfe3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c80c9740 0x7f78c8155b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45020/0 (socket says 172.21.15.40:45020) 2023-10-16T08:39:22.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78ccfe3640 1 -- 172.21.15.40:0/1881981343 learned_addr learned my addr 172.21.15.40:0/1881981343 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:22.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78bffff640 1 --2- 172.21.15.40:0/1881981343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f78c80ce030 0x7f78c814c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:22.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78cd7e4640 1 --2- 172.21.15.40:0/1881981343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78c80ce410 0x7f78c80c3860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:22.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78ccfe3640 1 -- 172.21.15.40:0/1881981343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f78c80ce030 msgr2=0x7f78c814c370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:22.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78ccfe3640 1 --2- 172.21.15.40:0/1881981343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f78c80ce030 0x7f78c814c370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78ccfe3640 1 -- 172.21.15.40:0/1881981343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78c80ce410 msgr2=0x7f78c80c3860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:22.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78ccfe3640 1 --2- 172.21.15.40:0/1881981343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78c80ce410 0x7f78c80c3860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78ccfe3640 1 -- 172.21.15.40:0/1881981343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78c400b770 con 0x7f78c80c9740 2023-10-16T08:39:22.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78bffff640 1 --2- 172.21.15.40:0/1881981343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f78c80ce030 0x7f78c814c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:22.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78cd7e4640 1 --2- 172.21.15.40:0/1881981343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78c80ce410 0x7f78c80c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:22.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78ccfe3640 1 --2- 172.21.15.40:0/1881981343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c80c9740 0x7f78c8155b40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f78c400b2c0 tx=0x7f78c401b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:22.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78bd7fa640 1 -- 172.21.15.40:0/1881981343 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c4006ee0 con 0x7f78c80c9740 2023-10-16T08:39:22.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78bd7fa640 1 -- 172.21.15.40:0/1881981343 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78c40b8620 con 0x7f78c80c9740 2023-10-16T08:39:22.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78cf247640 1 -- 172.21.15.40:0/1881981343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78c80c7fc0 con 0x7f78c80c9740 2023-10-16T08:39:22.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78bd7fa640 1 -- 172.21.15.40:0/1881981343 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78c40b89d0 con 0x7f78c80c9740 2023-10-16T08:39:22.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78cf247640 1 -- 172.21.15.40:0/1881981343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78c80c8580 con 0x7f78c80c9740 2023-10-16T08:39:22.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78cf247640 1 -- 172.21.15.40:0/1881981343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78c804e9c0 con 0x7f78c80c9740 2023-10-16T08:39:22.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78bd7fa640 1 -- 172.21.15.40:0/1881981343 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f78c40b6040 con 0x7f78c80c9740 2023-10-16T08:39:22.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78bd7fa640 1 --2- 172.21.15.40:0/1881981343 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f78b40664a0 0x7f78b4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:22.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.051+0000 7f78bd7fa640 1 -- 172.21.15.40:0/1881981343 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f78c4120ff0 con 0x7f78c80c9740 2023-10-16T08:39:22.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.055+0000 7f78bffff640 1 --2- 172.21.15.40:0/1881981343 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f78b40664a0 0x7f78b4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:22.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.055+0000 7f78bffff640 1 --2- 172.21.15.40:0/1881981343 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f78b40664a0 0x7f78b4068960 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f78c814fe90 tx=0x7f78b8008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:22.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.059+0000 7f78bd7fa640 1 -- 172.21.15.40:0/1881981343 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f78c40ee500 con 0x7f78c80c9740 2023-10-16T08:39:22.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.211+0000 7f78cf247640 1 -- 172.21.15.40:0/1881981343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f78c8073000 con 0x7f78c80c9740 2023-10-16T08:39:22.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.211+0000 7f78bd7fa640 1 -- 172.21.15.40:0/1881981343 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f78c40bc0a0 con 0x7f78c80c9740 2023-10-16T08:39:22.219 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-16T08:39:22.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.215+0000 7f78cf247640 1 -- 172.21.15.40:0/1881981343 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f78b40664a0 msgr2=0x7f78b4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:22.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.215+0000 7f78cf247640 1 --2- 172.21.15.40:0/1881981343 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f78b40664a0 0x7f78b4068960 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f78c814fe90 tx=0x7f78b8008040 comp rx=0 tx=0).stop 2023-10-16T08:39:22.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.215+0000 7f78cf247640 1 -- 172.21.15.40:0/1881981343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c80c9740 msgr2=0x7f78c8155b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:22.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.215+0000 7f78cf247640 1 --2- 172.21.15.40:0/1881981343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c80c9740 0x7f78c8155b40 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f78c400b2c0 tx=0x7f78c401b970 comp rx=0 tx=0).stop 2023-10-16T08:39:22.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.215+0000 7f78cf247640 1 -- 172.21.15.40:0/1881981343 shutdown_connections 2023-10-16T08:39:22.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.215+0000 7f78cf247640 1 --2- 172.21.15.40:0/1881981343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78c80ce410 0x7f78c80c3860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.215+0000 7f78cf247640 1 --2- 172.21.15.40:0/1881981343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f78c80ce030 0x7f78c814c370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.215+0000 7f78cf247640 1 --2- 172.21.15.40:0/1881981343 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f78b40664a0 0x7f78b4068960 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.215+0000 7f78cf247640 1 --2- 172.21.15.40:0/1881981343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f78c80c9740 0x7f78c8155b40 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.215+0000 7f78cf247640 1 -- 172.21.15.40:0/1881981343 >> 172.21.15.40:0/1881981343 conn(0x7f78c80cf0f0 msgr2=0x7f78c80b3170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:22.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.215+0000 7f78cf247640 1 -- 172.21.15.40:0/1881981343 shutdown_connections 2023-10-16T08:39:22.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.215+0000 7f78cf247640 1 -- 172.21.15.40:0/1881981343 wait complete. 2023-10-16T08:39:22.237 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:39:22.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.351+0000 7fbb8316c640 1 Processor -- start 2023-10-16T08:39:22.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.351+0000 7fbb8316c640 1 -- start start 2023-10-16T08:39:22.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.351+0000 7fbb8316c640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbb7c0ca2c0 0x7fbb7c0ca750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:22.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.351+0000 7fbb8316c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7c159fc0 con 0x7fbb7c159be0 2023-10-16T08:39:22.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.351+0000 7fbb8316c640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb7c06c450 con 0x7fbb7c0b1360 2023-10-16T08:39:22.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.351+0000 7fbb8316c640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb7c06c730 con 0x7fbb7c0ca2c0 2023-10-16T08:39:22.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.351+0000 7fbb80f08640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbb7c159be0 0x7fbb7c0cde30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60984/0 (socket says 172.21.15.40:60984) 2023-10-16T08:39:22.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.351+0000 7fbb80f08640 1 -- 172.21.15.40:0/240130337 learned_addr learned my addr 172.21.15.40:0/240130337 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:22.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.351+0000 7fbb7bfff640 1 --2- 172.21.15.40:0/240130337 >> v2:172.21.15.86:3300/0 conn(0x7fbb7c0ca2c0 0x7fbb7c0ca750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:22.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.351+0000 7fbb7b7fe640 1 -- 172.21.15.40:0/240130337 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2867687786 0 0) 0x7fbb7c159fc0 con 0x7fbb7c159be0 2023-10-16T08:39:22.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.351+0000 7fbb7b7fe640 1 -- 172.21.15.40:0/240130337 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb64005740 con 0x7fbb7c159be0 2023-10-16T08:39:22.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.351+0000 7fbb7b7fe640 1 -- 172.21.15.40:0/240130337 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2884061875 0 0) 0x7fbb64005740 con 0x7fbb7c159be0 2023-10-16T08:39:22.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.351+0000 7fbb7b7fe640 1 -- 172.21.15.40:0/240130337 >> v2:172.21.15.86:3300/0 conn(0x7fbb7c0ca2c0 msgr2=0x7fbb7c0ca750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:22.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.351+0000 7fbb7b7fe640 1 --2- 172.21.15.40:0/240130337 >> v2:172.21.15.86:3300/0 conn(0x7fbb7c0ca2c0 0x7fbb7c0ca750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.351+0000 7fbb7b7fe640 1 -- 172.21.15.40:0/240130337 >> v1:172.21.15.86:6789/0 conn(0x7fbb7c0b1360 legacy=0x7fbb7c06d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:22.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.351+0000 7fbb7b7fe640 1 -- 172.21.15.40:0/240130337 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c06ca10 con 0x7fbb7c159be0 2023-10-16T08:39:22.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.351+0000 7fbb7b7fe640 1 -- 172.21.15.40:0/240130337 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7fbb6c001680 con 0x7fbb7c159be0 2023-10-16T08:39:22.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb7b7fe640 1 -- 172.21.15.40:0/240130337 >> v1:172.21.15.40:6789/0 conn(0x7fbb7c159be0 legacy=0x7fbb7c0cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:22.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb7b7fe640 1 --2- 172.21.15.40:0/240130337 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb64005c00 0x7fbb64006020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:22.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb7b7fe640 1 --2- 172.21.15.40:0/240130337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb640069e0 0x7fbb64006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:22.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb7b7fe640 1 --2- 172.21.15.40:0/240130337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb64007410 0x7fbb6400dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:22.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb7b7fe640 1 -- 172.21.15.40:0/240130337 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c06c730 con 0x7fbb640069e0 2023-10-16T08:39:22.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb7b7fe640 1 -- 172.21.15.40:0/240130337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6400e370 con 0x7fbb64007410 2023-10-16T08:39:22.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb7b7fe640 1 -- 172.21.15.40:0/240130337 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6400e530 con 0x7fbb64005c00 2023-10-16T08:39:22.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb7bfff640 1 --2- 172.21.15.40:0/240130337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb640069e0 0x7fbb64006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:22.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb80f08640 1 --2- 172.21.15.40:0/240130337 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb64005c00 0x7fbb64006020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:22.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb7bfff640 1 -- 172.21.15.40:0/240130337 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb64005c00 msgr2=0x7fbb64006020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:22.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb7bfff640 1 --2- 172.21.15.40:0/240130337 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb64005c00 0x7fbb64006020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb7bfff640 1 -- 172.21.15.40:0/240130337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb64007410 msgr2=0x7fbb6400dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:22.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb7bfff640 1 --2- 172.21.15.40:0/240130337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb64007410 0x7fbb6400dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb7bfff640 1 -- 172.21.15.40:0/240130337 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb64005740 con 0x7fbb640069e0 2023-10-16T08:39:22.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb81709640 1 -- 172.21.15.40:0/240130337 reap_dead start 2023-10-16T08:39:22.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb7bfff640 1 --2- 172.21.15.40:0/240130337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb640069e0 0x7fbb64006e60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009ab0 tx=0x7fbb7001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f77c719c3bdee server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:22.370 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb7b7fe640 1 -- 172.21.15.40:0/240130337 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb700a5520 con 0x7fbb640069e0 2023-10-16T08:39:22.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb7b7fe640 1 -- 172.21.15.40:0/240130337 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb700a5680 con 0x7fbb640069e0 2023-10-16T08:39:22.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb7b7fe640 1 -- 172.21.15.40:0/240130337 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb700a5a30 con 0x7fbb640069e0 2023-10-16T08:39:22.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb8316c640 1 -- 172.21.15.40:0/240130337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb640069e0 msgr2=0x7fbb64006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:22.371 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb8316c640 1 --2- 172.21.15.40:0/240130337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb640069e0 0x7fbb64006e60 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fbb70009ab0 tx=0x7fbb7001ca40 comp rx=0 tx=0).stop 2023-10-16T08:39:22.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb8316c640 1 -- 172.21.15.40:0/240130337 shutdown_connections 2023-10-16T08:39:22.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb8316c640 1 --2- 172.21.15.40:0/240130337 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb640069e0 0x7fbb64006e60 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb8316c640 1 -- 172.21.15.40:0/240130337 >> 172.21.15.40:0/240130337 conn(0x7fbb7c0cc360 msgr2=0x7fbb7c0cc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:22.372 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb8316c640 1 -- 172.21.15.40:0/240130337 shutdown_connections 2023-10-16T08:39:22.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.355+0000 7fbb8316c640 1 -- 172.21.15.40:0/240130337 wait complete. 2023-10-16T08:39:22.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb8316c640 1 Processor -- start 2023-10-16T08:39:22.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb8316c640 1 -- start start 2023-10-16T08:39:22.373 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb8316c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb640069e0 0x7fbb7c152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:22.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb8316c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7c0c7a00 0x7fbb7c0c7e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:22.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb8316c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb7c15ab30 0x7fbb7c15afb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:22.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb8316c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c0c83c0 con 0x7fbb7c15ab30 2023-10-16T08:39:22.374 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb8316c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c15b8d0 con 0x7fbb7c0c7a00 2023-10-16T08:39:22.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb8316c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb7c15bbb0 con 0x7fbb640069e0 2023-10-16T08:39:22.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb81709640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb7c15ab30 0x7fbb7c15afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:22.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb81709640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb7c15ab30 0x7fbb7c15afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45042/0 (socket says 172.21.15.40:45042) 2023-10-16T08:39:22.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb81709640 1 -- 172.21.15.40:0/218256691 learned_addr learned my addr 172.21.15.40:0/218256691 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:22.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb7bfff640 1 --2- 172.21.15.40:0/218256691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7c0c7a00 0x7fbb7c0c7e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:22.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb80f08640 1 --2- 172.21.15.40:0/218256691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb640069e0 0x7fbb7c152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:22.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb81709640 1 -- 172.21.15.40:0/218256691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb640069e0 msgr2=0x7fbb7c152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:22.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb81709640 1 --2- 172.21.15.40:0/218256691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb640069e0 0x7fbb7c152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb81709640 1 -- 172.21.15.40:0/218256691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7c0c7a00 msgr2=0x7fbb7c0c7e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:22.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb81709640 1 --2- 172.21.15.40:0/218256691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7c0c7a00 0x7fbb7c0c7e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.377 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb81709640 1 -- 172.21.15.40:0/218256691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c0b2140 con 0x7fbb7c15ab30 2023-10-16T08:39:22.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb81709640 1 --2- 172.21.15.40:0/218256691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb7c15ab30 0x7fbb7c15afb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0c6960 tx=0x7fbb7400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:22.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb797fa640 1 -- 172.21.15.40:0/218256691 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74007070 con 0x7fbb7c15ab30 2023-10-16T08:39:22.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb797fa640 1 -- 172.21.15.40:0/218256691 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb740071d0 con 0x7fbb7c15ab30 2023-10-16T08:39:22.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb8316c640 1 -- 172.21.15.40:0/218256691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb7c0b2420 con 0x7fbb7c15ab30 2023-10-16T08:39:22.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb797fa640 1 -- 172.21.15.40:0/218256691 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb74007580 con 0x7fbb7c15ab30 2023-10-16T08:39:22.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb8316c640 1 -- 172.21.15.40:0/218256691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb7c152c60 con 0x7fbb7c15ab30 2023-10-16T08:39:22.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.359+0000 7fbb8316c640 1 -- 172.21.15.40:0/218256691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb7c04e9c0 con 0x7fbb7c15ab30 2023-10-16T08:39:22.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.363+0000 7fbb797fa640 1 -- 172.21.15.40:0/218256691 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbb740076e0 con 0x7fbb7c15ab30 2023-10-16T08:39:22.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.363+0000 7fbb797fa640 1 --2- 172.21.15.40:0/218256691 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbb680664a0 0x7fbb68068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:22.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.363+0000 7fbb797fa640 1 -- 172.21.15.40:0/218256691 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb740832b0 con 0x7fbb7c15ab30 2023-10-16T08:39:22.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.363+0000 7fbb80f08640 1 --2- 172.21.15.40:0/218256691 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbb680664a0 0x7fbb68068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:22.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.363+0000 7fbb80f08640 1 --2- 172.21.15.40:0/218256691 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbb680664a0 0x7fbb68068960 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c000fc0 tx=0x7fbb6c001540 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:22.382 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.367+0000 7fbb797fa640 1 -- 172.21.15.40:0/218256691 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fbb74050730 con 0x7fbb7c15ab30 2023-10-16T08:39:22.550 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.543+0000 7fbb8316c640 1 -- 172.21.15.40:0/218256691 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbb7c070810 con 0x7fbb7c15ab30 2023-10-16T08:39:22.551 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.543+0000 7fbb797fa640 1 -- 172.21.15.40:0/218256691 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbb74017080 con 0x7fbb7c15ab30 2023-10-16T08:39:22.551 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-16T08:39:22.552 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.547+0000 7fbb8316c640 1 -- 172.21.15.40:0/218256691 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbb680664a0 msgr2=0x7fbb68068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:22.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.547+0000 7fbb8316c640 1 --2- 172.21.15.40:0/218256691 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbb680664a0 0x7fbb68068960 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fbb6c000fc0 tx=0x7fbb6c001540 comp rx=0 tx=0).stop 2023-10-16T08:39:22.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.547+0000 7fbb8316c640 1 -- 172.21.15.40:0/218256691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb7c15ab30 msgr2=0x7fbb7c15afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:22.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.547+0000 7fbb8316c640 1 --2- 172.21.15.40:0/218256691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb7c15ab30 0x7fbb7c15afb0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0c6960 tx=0x7fbb7400c960 comp rx=0 tx=0).stop 2023-10-16T08:39:22.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.547+0000 7fbb8316c640 1 -- 172.21.15.40:0/218256691 shutdown_connections 2023-10-16T08:39:22.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.547+0000 7fbb8316c640 1 --2- 172.21.15.40:0/218256691 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbb680664a0 0x7fbb68068960 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.547+0000 7fbb8316c640 1 --2- 172.21.15.40:0/218256691 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbb7c15ab30 0x7fbb7c15afb0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.547+0000 7fbb8316c640 1 --2- 172.21.15.40:0/218256691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7c0c7a00 0x7fbb7c0c7e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.547+0000 7fbb8316c640 1 --2- 172.21.15.40:0/218256691 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbb640069e0 0x7fbb7c152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.547+0000 7fbb8316c640 1 -- 172.21.15.40:0/218256691 >> 172.21.15.40:0/218256691 conn(0x7fbb7c0cc360 msgr2=0x7fbb7c0cb7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:22.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.547+0000 7fbb8316c640 1 -- 172.21.15.40:0/218256691 shutdown_connections 2023-10-16T08:39:22.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.547+0000 7fbb8316c640 1 -- 172.21.15.40:0/218256691 wait complete. 2023-10-16T08:39:22.570 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi040.front.sepia.ceph.com... 2023-10-16T08:39:22.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:22.571 DEBUG:teuthology.orchestra.run.smithi040:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-10-16T08:39:22.583 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi040.stderr:ceph-fuse[45215]: fuse finished with error 0 and tester_r 0 2023-10-16T08:39:22.584 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:39:22.624 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi040.front.sepia.ceph.com 2023-10-16T08:39:22.625 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:22.625 DEBUG:teuthology.orchestra.run.smithi040:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-16T08:39:22.672 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:39:22.672 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:39:22.679 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:39:22.680 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:39:22.730 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:39:22.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.851+0000 7fdd5b044640 1 Processor -- start 2023-10-16T08:39:22.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.855+0000 7fdd5b044640 1 -- start start 2023-10-16T08:39:22.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.855+0000 7fdd5b044640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdd5415d560 0x7fdd5406b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:22.861 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.855+0000 7fdd5b044640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdd540ad820 0x7fdd540c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:22.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.855+0000 7fdd5b044640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd540cf480 con 0x7fdd5415d180 2023-10-16T08:39:22.862 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.855+0000 7fdd5b044640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd540cf760 con 0x7fdd540ad820 2023-10-16T08:39:22.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.855+0000 7fdd5b044640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd540c8fb0 con 0x7fdd5415d560 2023-10-16T08:39:22.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.855+0000 7fdd58de0640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fdd5415d180 0x7fdd540c9570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:60998/0 (socket says 172.21.15.40:60998) 2023-10-16T08:39:22.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.855+0000 7fdd58de0640 1 -- 172.21.15.40:0/1482730956 learned_addr learned my addr 172.21.15.40:0/1482730956 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:22.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.855+0000 7fdd595e1640 1 --2- 172.21.15.40:0/1482730956 >> v2:172.21.15.40:3300/0 conn(0x7fdd540ad820 0x7fdd540c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:22.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.855+0000 7fdd53fff640 1 --2- 172.21.15.40:0/1482730956 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdd5415d560 0x7fdd5406b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:22.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.855+0000 7fdd595e1640 1 -- 172.21.15.40:0/1482730956 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdd5415d560 msgr2=0x7fdd5406b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:22.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.855+0000 7fdd595e1640 1 --2- 172.21.15.40:0/1482730956 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdd5415d560 0x7fdd5406b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.855+0000 7fdd595e1640 1 -- 172.21.15.40:0/1482730956 >> v1:172.21.15.40:6789/0 conn(0x7fdd5415d180 legacy=0x7fdd540c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:22.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.855+0000 7fdd595e1640 1 -- 172.21.15.40:0/1482730956 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd540c9290 con 0x7fdd540ad820 2023-10-16T08:39:22.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.855+0000 7fdd595e1640 1 --2- 172.21.15.40:0/1482730956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd540ad820 0x7fdd540c9c80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0097c0 tx=0x7fdd4c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24fb28afb66c3a76 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:22.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.855+0000 7fdd537fe640 1 -- 172.21.15.40:0/1482730956 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c0a6040 con 0x7fdd540ad820 2023-10-16T08:39:22.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd537fe640 1 -- 172.21.15.40:0/1482730956 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4c0aa600 con 0x7fdd540ad820 2023-10-16T08:39:22.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd537fe640 1 -- 172.21.15.40:0/1482730956 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4c0aa990 con 0x7fdd540ad820 2023-10-16T08:39:22.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd5b044640 1 -- 172.21.15.40:0/1482730956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd540ad820 msgr2=0x7fdd540c9c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:22.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd5b044640 1 --2- 172.21.15.40:0/1482730956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd540ad820 0x7fdd540c9c80 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fdd4c0097c0 tx=0x7fdd4c01c480 comp rx=0 tx=0).stop 2023-10-16T08:39:22.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd5b044640 1 -- 172.21.15.40:0/1482730956 shutdown_connections 2023-10-16T08:39:22.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd5b044640 1 --2- 172.21.15.40:0/1482730956 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd540ad820 0x7fdd540c9c80 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd5b044640 1 --2- 172.21.15.40:0/1482730956 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdd5415d560 0x7fdd5406b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd5b044640 1 -- 172.21.15.40:0/1482730956 >> 172.21.15.40:0/1482730956 conn(0x7fdd540ce960 msgr2=0x7fdd540ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:22.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd5b044640 1 -- 172.21.15.40:0/1482730956 shutdown_connections 2023-10-16T08:39:22.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd5b044640 1 -- 172.21.15.40:0/1482730956 wait complete. 2023-10-16T08:39:22.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd5b044640 1 Processor -- start 2023-10-16T08:39:22.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd5b044640 1 -- start start 2023-10-16T08:39:22.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd5b044640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd540ad820 0x7fdd54151ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:22.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd5b044640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd5415d180 0x7fdd540c47d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:22.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd5b044640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdd5415d560 0x7fdd54156ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:22.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd5b044640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd540c5520 con 0x7fdd5415d180 2023-10-16T08:39:22.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd5b044640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd541520e0 con 0x7fdd540ad820 2023-10-16T08:39:22.874 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd5b044640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd540b1d70 con 0x7fdd5415d560 2023-10-16T08:39:22.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd53fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd5415d180 0x7fdd540c47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:22.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd53fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd5415d180 0x7fdd540c47d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45062/0 (socket says 172.21.15.40:45062) 2023-10-16T08:39:22.875 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd53fff640 1 -- 172.21.15.40:0/3012597013 learned_addr learned my addr 172.21.15.40:0/3012597013 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:22.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd58de0640 1 --2- 172.21.15.40:0/3012597013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd540ad820 0x7fdd54151ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:22.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.859+0000 7fdd595e1640 1 --2- 172.21.15.40:0/3012597013 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdd5415d560 0x7fdd54156ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:22.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.863+0000 7fdd53fff640 1 -- 172.21.15.40:0/3012597013 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdd5415d560 msgr2=0x7fdd54156ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:22.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.863+0000 7fdd53fff640 1 --2- 172.21.15.40:0/3012597013 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdd5415d560 0x7fdd54156ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.863+0000 7fdd53fff640 1 -- 172.21.15.40:0/3012597013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd540ad820 msgr2=0x7fdd54151ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:22.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.863+0000 7fdd53fff640 1 --2- 172.21.15.40:0/3012597013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd540ad820 0x7fdd54151ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:22.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.863+0000 7fdd53fff640 1 -- 172.21.15.40:0/3012597013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd4c009420 con 0x7fdd5415d180 2023-10-16T08:39:22.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.863+0000 7fdd53fff640 1 --2- 172.21.15.40:0/3012597013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd5415d180 0x7fdd540c47d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c007e40 tx=0x7fdd3c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:22.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.863+0000 7fdd517fa640 1 -- 172.21.15.40:0/3012597013 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c014620 con 0x7fdd5415d180 2023-10-16T08:39:22.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.863+0000 7fdd517fa640 1 -- 172.21.15.40:0/3012597013 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd3c00ee90 con 0x7fdd5415d180 2023-10-16T08:39:22.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.863+0000 7fdd517fa640 1 -- 172.21.15.40:0/3012597013 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd3c0149c0 con 0x7fdd5415d180 2023-10-16T08:39:22.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.863+0000 7fdd5b044640 1 -- 172.21.15.40:0/3012597013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd540b34b0 con 0x7fdd5415d180 2023-10-16T08:39:22.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.863+0000 7fdd5b044640 1 -- 172.21.15.40:0/3012597013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd540b1720 con 0x7fdd5415d180 2023-10-16T08:39:22.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.863+0000 7fdd5b044640 1 -- 172.21.15.40:0/3012597013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd540b1580 con 0x7fdd5415d180 2023-10-16T08:39:22.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.863+0000 7fdd517fa640 1 -- 172.21.15.40:0/3012597013 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdd3c014b20 con 0x7fdd5415d180 2023-10-16T08:39:22.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.863+0000 7fdd517fa640 1 --2- 172.21.15.40:0/3012597013 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdd340664d0 0x7fdd34068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:22.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.863+0000 7fdd517fa640 1 -- 172.21.15.40:0/3012597013 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdd3c084730 con 0x7fdd5415d180 2023-10-16T08:39:22.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.863+0000 7fdd58de0640 1 --2- 172.21.15.40:0/3012597013 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdd340664d0 0x7fdd34068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:22.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.863+0000 7fdd58de0640 1 --2- 172.21.15.40:0/3012597013 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdd340664d0 0x7fdd34068990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdd44005890 tx=0x7fdd44009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:22.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:22.867+0000 7fdd517fa640 1 -- 172.21.15.40:0/3012597013 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fdd540b1580 con 0x7fdd5415d180 2023-10-16T08:39:23.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.019+0000 7fdd5b044640 1 -- 172.21.15.40:0/3012597013 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdd540b1580 con 0x7fdd5415d180 2023-10-16T08:39:23.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.023+0000 7fdd517fa640 1 -- 172.21.15.40:0/3012597013 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16445 (secure 0 0 0) 0x7fdd540b1580 con 0x7fdd5415d180 2023-10-16T08:39:23.028 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:39:23.028 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:46.611098+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:39:23.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.023+0000 7fdd5b044640 1 -- 172.21.15.40:0/3012597013 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdd340664d0 msgr2=0x7fdd34068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:23.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.023+0000 7fdd5b044640 1 --2- 172.21.15.40:0/3012597013 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdd340664d0 0x7fdd34068990 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fdd44005890 tx=0x7fdd44009040 comp rx=0 tx=0).stop 2023-10-16T08:39:23.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.023+0000 7fdd5b044640 1 -- 172.21.15.40:0/3012597013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd5415d180 msgr2=0x7fdd540c47d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:23.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.023+0000 7fdd5b044640 1 --2- 172.21.15.40:0/3012597013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd5415d180 0x7fdd540c47d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fdd3c007e40 tx=0x7fdd3c00e4b0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.023+0000 7fdd5b044640 1 -- 172.21.15.40:0/3012597013 shutdown_connections 2023-10-16T08:39:23.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.023+0000 7fdd5b044640 1 --2- 172.21.15.40:0/3012597013 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdd5415d560 0x7fdd54156ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.023+0000 7fdd5b044640 1 --2- 172.21.15.40:0/3012597013 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdd340664d0 0x7fdd34068990 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.023+0000 7fdd5b044640 1 --2- 172.21.15.40:0/3012597013 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdd5415d180 0x7fdd540c47d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.023+0000 7fdd5b044640 1 --2- 172.21.15.40:0/3012597013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd540ad820 0x7fdd54151ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.023+0000 7fdd5b044640 1 -- 172.21.15.40:0/3012597013 >> 172.21.15.40:0/3012597013 conn(0x7fdd540ce960 msgr2=0x7fdd540ae270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:23.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.023+0000 7fdd5b044640 1 -- 172.21.15.40:0/3012597013 shutdown_connections 2023-10-16T08:39:23.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.023+0000 7fdd5b044640 1 -- 172.21.15.40:0/3012597013 wait complete. 2023-10-16T08:39:23.045 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:38:33.014668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:38:43.055215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:38:43.971666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:39:23.046 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:39:23.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.167+0000 7f11276bf640 1 Processor -- start 2023-10-16T08:39:23.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.167+0000 7f11276bf640 1 -- start start 2023-10-16T08:39:23.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.167+0000 7f11276bf640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1120154040 0x7f1120072d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:23.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.167+0000 7f11276bf640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11200ce450 0x7f112006fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:23.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.167+0000 7f11276bf640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f112006c920 con 0x7f112006e360 2023-10-16T08:39:23.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.167+0000 7f11276bf640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f112006cb90 con 0x7f1120154040 2023-10-16T08:39:23.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.167+0000 7f11276bf640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112006ce70 con 0x7f11200ce450 2023-10-16T08:39:23.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.167+0000 7f112545b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1120154040 0x7f1120072d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:23.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.167+0000 7f112545b640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1120154040 0x7f1120072d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45072/0 (socket says 172.21.15.40:45072) 2023-10-16T08:39:23.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.167+0000 7f112545b640 1 -- 172.21.15.40:0/1477419430 learned_addr learned my addr 172.21.15.40:0/1477419430 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:23.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.167+0000 7f1125c5c640 1 --2- 172.21.15.40:0/1477419430 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11200ce450 0x7f112006fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:23.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.167+0000 7f112545b640 1 -- 172.21.15.40:0/1477419430 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11200ce450 msgr2=0x7f112006fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:23.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.167+0000 7f112545b640 1 --2- 172.21.15.40:0/1477419430 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11200ce450 0x7f112006fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.167+0000 7f112545b640 1 -- 172.21.15.40:0/1477419430 >> v1:172.21.15.86:6789/0 conn(0x7f112006e360 legacy=0x7f11200cdd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:23.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.167+0000 7f112545b640 1 -- 172.21.15.40:0/1477419430 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112006d150 con 0x7f1120154040 2023-10-16T08:39:23.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f112545b640 1 --2- 172.21.15.40:0/1477419430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1120154040 0x7f1120072d60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f111c009870 tx=0x7f111c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce6f40309a04c3c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:23.179 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f1117fff640 1 -- 172.21.15.40:0/1477419430 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c0ab600 con 0x7f1120154040 2023-10-16T08:39:23.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f1117fff640 1 -- 172.21.15.40:0/1477419430 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f111c0a7050 con 0x7f1120154040 2023-10-16T08:39:23.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f1117fff640 1 -- 172.21.15.40:0/1477419430 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f111c0ab980 con 0x7f1120154040 2023-10-16T08:39:23.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f11276bf640 1 -- 172.21.15.40:0/1477419430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1120154040 msgr2=0x7f1120072d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:23.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f11276bf640 1 --2- 172.21.15.40:0/1477419430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1120154040 0x7f1120072d60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f111c009870 tx=0x7f111c01c4d0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f11276bf640 1 -- 172.21.15.40:0/1477419430 shutdown_connections 2023-10-16T08:39:23.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f11276bf640 1 --2- 172.21.15.40:0/1477419430 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f11200ce450 0x7f112006fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f11276bf640 1 --2- 172.21.15.40:0/1477419430 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1120154040 0x7f1120072d60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f11276bf640 1 -- 172.21.15.40:0/1477419430 >> 172.21.15.40:0/1477419430 conn(0x7f11200cf060 msgr2=0x7f11200cf470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:23.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f11276bf640 1 -- 172.21.15.40:0/1477419430 shutdown_connections 2023-10-16T08:39:23.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f11276bf640 1 -- 172.21.15.40:0/1477419430 wait complete. 2023-10-16T08:39:23.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f11276bf640 1 Processor -- start 2023-10-16T08:39:23.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f11276bf640 1 -- start start 2023-10-16T08:39:23.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f11276bf640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f112006e360 0x7f112015bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:23.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f11276bf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11200ce450 0x7f1120155d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:23.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f11276bf640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1120154040 0x7f1120156290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:23.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f11276bf640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1120154f40 con 0x7f1120154040 2023-10-16T08:39:23.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f11276bf640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11200c23a0 con 0x7f11200ce450 2023-10-16T08:39:23.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f11276bf640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112015c5b0 con 0x7f112006e360 2023-10-16T08:39:23.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f1125c5c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1120154040 0x7f1120156290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:23.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f1125c5c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1120154040 0x7f1120156290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45084/0 (socket says 172.21.15.40:45084) 2023-10-16T08:39:23.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f1125c5c640 1 -- 172.21.15.40:0/1956661896 learned_addr learned my addr 172.21.15.40:0/1956661896 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:23.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f112545b640 1 --2- 172.21.15.40:0/1956661896 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f112006e360 0x7f112015bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:23.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f1124c5a640 1 --2- 172.21.15.40:0/1956661896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11200ce450 0x7f1120155d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:23.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f1125c5c640 1 -- 172.21.15.40:0/1956661896 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f112006e360 msgr2=0x7f112015bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:23.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f1125c5c640 1 --2- 172.21.15.40:0/1956661896 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f112006e360 0x7f112015bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f1125c5c640 1 -- 172.21.15.40:0/1956661896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11200ce450 msgr2=0x7f1120155d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:23.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f1125c5c640 1 --2- 172.21.15.40:0/1956661896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11200ce450 0x7f1120155d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f1125c5c640 1 -- 172.21.15.40:0/1956661896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f111c0094d0 con 0x7f1120154040 2023-10-16T08:39:23.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.171+0000 7f1125c5c640 1 --2- 172.21.15.40:0/1956661896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1120154040 0x7f1120156290 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f111800cd60 tx=0x7f111800a660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:23.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.175+0000 7f1115ffb640 1 -- 172.21.15.40:0/1956661896 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1118003ba0 con 0x7f1120154040 2023-10-16T08:39:23.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.175+0000 7f1115ffb640 1 -- 172.21.15.40:0/1956661896 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1118003d20 con 0x7f1120154040 2023-10-16T08:39:23.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.175+0000 7f11276bf640 1 -- 172.21.15.40:0/1956661896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f112015c890 con 0x7f1120154040 2023-10-16T08:39:23.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.175+0000 7f1115ffb640 1 -- 172.21.15.40:0/1956661896 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1118004260 con 0x7f1120154040 2023-10-16T08:39:23.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.175+0000 7f11276bf640 1 -- 172.21.15.40:0/1956661896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f112015ce50 con 0x7f1120154040 2023-10-16T08:39:23.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.175+0000 7f11276bf640 1 -- 172.21.15.40:0/1956661896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f112013e110 con 0x7f1120154040 2023-10-16T08:39:23.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.175+0000 7f1115ffb640 1 -- 172.21.15.40:0/1956661896 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f11180043c0 con 0x7f1120154040 2023-10-16T08:39:23.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.175+0000 7f1115ffb640 1 --2- 172.21.15.40:0/1956661896 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f11040664a0 0x7f1104068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:23.190 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.175+0000 7f1115ffb640 1 -- 172.21.15.40:0/1956661896 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1118012070 con 0x7f1120154040 2023-10-16T08:39:23.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.175+0000 7f112545b640 1 --2- 172.21.15.40:0/1956661896 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f11040664a0 0x7f1104068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:23.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.175+0000 7f112545b640 1 --2- 172.21.15.40:0/1956661896 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f11040664a0 0x7f1104068960 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f111c0ad000 tx=0x7f111c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:23.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.179+0000 7f1115ffb640 1 -- 172.21.15.40:0/1956661896 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f1118051c70 con 0x7f1120154040 2023-10-16T08:39:23.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.335+0000 7f11276bf640 1 -- 172.21.15.40:0/1956661896 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f112004e9c0 con 0x7f1120154040 2023-10-16T08:39:23.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.339+0000 7f1115ffb640 1 -- 172.21.15.40:0/1956661896 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1118055290 con 0x7f1120154040 2023-10-16T08:39:23.344 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-16T08:39:23.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.339+0000 7f11276bf640 1 -- 172.21.15.40:0/1956661896 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f11040664a0 msgr2=0x7f1104068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:23.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.339+0000 7f11276bf640 1 --2- 172.21.15.40:0/1956661896 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f11040664a0 0x7f1104068960 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f111c0ad000 tx=0x7f111c008040 comp rx=0 tx=0).stop 2023-10-16T08:39:23.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.339+0000 7f11276bf640 1 -- 172.21.15.40:0/1956661896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1120154040 msgr2=0x7f1120156290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:23.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.339+0000 7f11276bf640 1 --2- 172.21.15.40:0/1956661896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1120154040 0x7f1120156290 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f111800cd60 tx=0x7f111800a660 comp rx=0 tx=0).stop 2023-10-16T08:39:23.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.339+0000 7f11276bf640 1 -- 172.21.15.40:0/1956661896 shutdown_connections 2023-10-16T08:39:23.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.339+0000 7f11276bf640 1 --2- 172.21.15.40:0/1956661896 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f11040664a0 0x7f1104068960 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.339+0000 7f11276bf640 1 --2- 172.21.15.40:0/1956661896 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1120154040 0x7f1120156290 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.339+0000 7f11276bf640 1 --2- 172.21.15.40:0/1956661896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11200ce450 0x7f1120155d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.339+0000 7f11276bf640 1 --2- 172.21.15.40:0/1956661896 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f112006e360 0x7f112015bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.339+0000 7f11276bf640 1 -- 172.21.15.40:0/1956661896 >> 172.21.15.40:0/1956661896 conn(0x7f11200cf060 msgr2=0x7f11200cf440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:23.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.339+0000 7f11276bf640 1 -- 172.21.15.40:0/1956661896 shutdown_connections 2023-10-16T08:39:23.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.339+0000 7f11276bf640 1 -- 172.21.15.40:0/1956661896 wait complete. 2023-10-16T08:39:23.362 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:39:23.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.487+0000 7f007c47f640 1 Processor -- start 2023-10-16T08:39:23.492 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.487+0000 7f007c47f640 1 -- start start 2023-10-16T08:39:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.487+0000 7f007c47f640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f007415cdc0 0x7f00740d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.487+0000 7f007c47f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f00740d3310 0x7f00740b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:23.494 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.487+0000 7f007c47f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00740d1870 con 0x7f00740b8590 2023-10-16T08:39:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.487+0000 7f007c47f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00740d1b50 con 0x7f00740d3310 2023-10-16T08:39:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.487+0000 7f007c47f640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00740d1e30 con 0x7f007415cdc0 2023-10-16T08:39:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.487+0000 7f007aa1c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f00740d3310 0x7f00740b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:23.495 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.487+0000 7f007aa1c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f00740d3310 0x7f00740b5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45096/0 (socket says 172.21.15.40:45096) 2023-10-16T08:39:23.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.487+0000 7f0079a1a640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f00740b8590 0x7f00740d2c00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32776/0 (socket says 172.21.15.40:32776) 2023-10-16T08:39:23.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.487+0000 7f007aa1c640 1 -- 172.21.15.40:0/1324673330 learned_addr learned my addr 172.21.15.40:0/1324673330 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:23.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.487+0000 7f007a21b640 1 --2- 172.21.15.40:0/1324673330 >> v2:172.21.15.86:3300/0 conn(0x7f007415cdc0 0x7f00740d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:23.496 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.487+0000 7f007aa1c640 1 -- 172.21.15.40:0/1324673330 >> v2:172.21.15.86:3300/0 conn(0x7f007415cdc0 msgr2=0x7f00740d1040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:23.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.487+0000 7f007aa1c640 1 --2- 172.21.15.40:0/1324673330 >> v2:172.21.15.86:3300/0 conn(0x7f007415cdc0 0x7f00740d1040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.487+0000 7f0079219640 1 -- 172.21.15.40:0/1324673330 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2675164929 0 0) 0x7f00740d1870 con 0x7f00740b8590 2023-10-16T08:39:23.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.487+0000 7f007aa1c640 1 -- 172.21.15.40:0/1324673330 >> v1:172.21.15.40:6789/0 conn(0x7f00740b8590 legacy=0x7f00740d2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:23.497 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.487+0000 7f007aa1c640 1 -- 172.21.15.40:0/1324673330 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00740d2110 con 0x7f00740d3310 2023-10-16T08:39:23.500 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f007aa1c640 1 --2- 172.21.15.40:0/1324673330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00740d3310 0x7f00740b5da0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f00740d0610 tx=0x7f006c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a429427a910aebf server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:23.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f0079219640 1 -- 172.21.15.40:0/1324673330 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c0a5840 con 0x7f00740d3310 2023-10-16T08:39:23.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f0079219640 1 -- 172.21.15.40:0/1324673330 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c0a59a0 con 0x7f00740d3310 2023-10-16T08:39:23.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f0079219640 1 -- 172.21.15.40:0/1324673330 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c0a5d30 con 0x7f00740d3310 2023-10-16T08:39:23.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f007c47f640 1 -- 172.21.15.40:0/1324673330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00740d3310 msgr2=0x7f00740b5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:23.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f007c47f640 1 --2- 172.21.15.40:0/1324673330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00740d3310 0x7f00740b5da0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f00740d0610 tx=0x7f006c01c9a0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f007c47f640 1 -- 172.21.15.40:0/1324673330 shutdown_connections 2023-10-16T08:39:23.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f007c47f640 1 --2- 172.21.15.40:0/1324673330 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00740d3310 0x7f00740b5da0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f007c47f640 1 --2- 172.21.15.40:0/1324673330 >> v2:172.21.15.86:3300/0 conn(0x7f007415cdc0 0x7f00740d1040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f007c47f640 1 -- 172.21.15.40:0/1324673330 >> 172.21.15.40:0/1324673330 conn(0x7f007406a290 msgr2=0x7f007406a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:23.503 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f007c47f640 1 -- 172.21.15.40:0/1324673330 shutdown_connections 2023-10-16T08:39:23.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f007c47f640 1 -- 172.21.15.40:0/1324673330 wait complete. 2023-10-16T08:39:23.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f007c47f640 1 Processor -- start 2023-10-16T08:39:23.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f007c47f640 1 -- start start 2023-10-16T08:39:23.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f007c47f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00740b8590 0x7f007415e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:23.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f007c47f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f00740d3310 0x7f007414d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:23.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f007c47f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f007415cdc0 0x7f00741539d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:23.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f007c47f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007415eff0 con 0x7f007415cdc0 2023-10-16T08:39:23.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f007c47f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007415f2d0 con 0x7f00740b8590 2023-10-16T08:39:23.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f007c47f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007415f5b0 con 0x7f00740d3310 2023-10-16T08:39:23.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f007aa1c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f007415cdc0 0x7f00741539d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:23.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f007aa1c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f007415cdc0 0x7f00741539d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45102/0 (socket says 172.21.15.40:45102) 2023-10-16T08:39:23.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.491+0000 7f007aa1c640 1 -- 172.21.15.40:0/3735210798 learned_addr learned my addr 172.21.15.40:0/3735210798 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:23.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.495+0000 7f007a21b640 1 --2- 172.21.15.40:0/3735210798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00740b8590 0x7f007415e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:23.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.495+0000 7f007aa1c640 1 -- 172.21.15.40:0/3735210798 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f00740d3310 msgr2=0x7f007414d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:23.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.495+0000 7f0079a1a640 1 --2- 172.21.15.40:0/3735210798 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f00740d3310 0x7f007414d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:23.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.495+0000 7f007aa1c640 1 --2- 172.21.15.40:0/3735210798 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f00740d3310 0x7f007414d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.495+0000 7f007aa1c640 1 -- 172.21.15.40:0/3735210798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00740b8590 msgr2=0x7f007415e0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:23.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.495+0000 7f007aa1c640 1 --2- 172.21.15.40:0/3735210798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00740b8590 0x7f007415e0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.495+0000 7f007aa1c640 1 -- 172.21.15.40:0/3735210798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f006c009b30 con 0x7f007415cdc0 2023-10-16T08:39:23.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.495+0000 7f007aa1c640 1 --2- 172.21.15.40:0/3735210798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f007415cdc0 0x7f00741539d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f006c0af040 tx=0x7f006c019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:23.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.495+0000 7f006affd640 1 -- 172.21.15.40:0/3735210798 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c0b8670 con 0x7f007415cdc0 2023-10-16T08:39:23.509 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.495+0000 7f006affd640 1 -- 172.21.15.40:0/3735210798 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f006c0b87d0 con 0x7f007415cdc0 2023-10-16T08:39:23.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.495+0000 7f007c47f640 1 -- 172.21.15.40:0/3735210798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f007415f890 con 0x7f007415cdc0 2023-10-16T08:39:23.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.495+0000 7f006affd640 1 -- 172.21.15.40:0/3735210798 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f006c0b8b80 con 0x7f007415cdc0 2023-10-16T08:39:23.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.495+0000 7f007c47f640 1 -- 172.21.15.40:0/3735210798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007415e600 con 0x7f007415cdc0 2023-10-16T08:39:23.510 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.495+0000 7f007c47f640 1 -- 172.21.15.40:0/3735210798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f007404e9c0 con 0x7f007415cdc0 2023-10-16T08:39:23.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.495+0000 7f006affd640 1 -- 172.21.15.40:0/3735210798 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f006c0b8ce0 con 0x7f007415cdc0 2023-10-16T08:39:23.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.495+0000 7f006affd640 1 --2- 172.21.15.40:0/3735210798 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f00580664a0 0x7f0058068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:23.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.495+0000 7f006affd640 1 -- 172.21.15.40:0/3735210798 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f006c11f3f0 con 0x7f007415cdc0 2023-10-16T08:39:23.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.495+0000 7f007a21b640 1 --2- 172.21.15.40:0/3735210798 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f00580664a0 0x7f0058068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:23.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.499+0000 7f007a21b640 1 --2- 172.21.15.40:0/3735210798 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f00580664a0 0x7f0058068960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0070009a50 tx=0x7f0070008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:23.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.499+0000 7f006affd640 1 -- 172.21.15.40:0/3735210798 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f006c0eca40 con 0x7f007415cdc0 2023-10-16T08:39:23.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.655+0000 7f007c47f640 1 -- 172.21.15.40:0/3735210798 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f007415f6f0 con 0x7f007415cdc0 2023-10-16T08:39:23.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.655+0000 7f006affd640 1 -- 172.21.15.40:0/3735210798 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7f007415f6f0 con 0x7f007415cdc0 2023-10-16T08:39:23.662 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-16T08:39:23.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.659+0000 7f007c47f640 1 -- 172.21.15.40:0/3735210798 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f00580664a0 msgr2=0x7f0058068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:23.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.659+0000 7f007c47f640 1 --2- 172.21.15.40:0/3735210798 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f00580664a0 0x7f0058068960 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0070009a50 tx=0x7f0070008040 comp rx=0 tx=0).stop 2023-10-16T08:39:23.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.659+0000 7f007c47f640 1 -- 172.21.15.40:0/3735210798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f007415cdc0 msgr2=0x7f00741539d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:23.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.659+0000 7f007c47f640 1 --2- 172.21.15.40:0/3735210798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f007415cdc0 0x7f00741539d0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f006c0af040 tx=0x7f006c019740 comp rx=0 tx=0).stop 2023-10-16T08:39:23.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.659+0000 7f007c47f640 1 -- 172.21.15.40:0/3735210798 shutdown_connections 2023-10-16T08:39:23.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.659+0000 7f007c47f640 1 --2- 172.21.15.40:0/3735210798 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f00580664a0 0x7f0058068960 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.659+0000 7f007c47f640 1 --2- 172.21.15.40:0/3735210798 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f007415cdc0 0x7f00741539d0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.659+0000 7f007c47f640 1 --2- 172.21.15.40:0/3735210798 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f00740d3310 0x7f007414d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.659+0000 7f007c47f640 1 --2- 172.21.15.40:0/3735210798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00740b8590 0x7f007415e0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.659+0000 7f007c47f640 1 -- 172.21.15.40:0/3735210798 >> 172.21.15.40:0/3735210798 conn(0x7f007406a290 msgr2=0x7f00740b77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:23.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.659+0000 7f007c47f640 1 -- 172.21.15.40:0/3735210798 shutdown_connections 2023-10-16T08:39:23.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.659+0000 7f007c47f640 1 -- 172.21.15.40:0/3735210798 wait complete. 2023-10-16T08:39:23.681 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:39:23.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef3c0f640 1 Processor -- start 2023-10-16T08:39:23.804 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef3c0f640 1 -- start start 2023-10-16T08:39:23.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef3c0f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3eec0afb90 0x7f3eec0cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:23.807 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef3c0f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eec0cd2e0 con 0x7f3eec0d00f0 2023-10-16T08:39:23.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef3c0f640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eec0cf930 con 0x7f3eec0ce540 2023-10-16T08:39:23.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef3c0f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec0cde80 con 0x7f3eec0afb90 2023-10-16T08:39:23.808 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef21ac640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3eec0d00f0 0x7f3eec164750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32786/0 (socket says 172.21.15.40:32786) 2023-10-16T08:39:23.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef21ac640 1 -- 172.21.15.40:0/1328288731 learned_addr learned my addr 172.21.15.40:0/1328288731 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:23.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef11aa640 1 --2- 172.21.15.40:0/1328288731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3eec0afb90 0x7f3eec0cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:23.809 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef09a9640 1 -- 172.21.15.40:0/1328288731 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2598028030 0 0) 0x7f3eec0cd2e0 con 0x7f3eec0d00f0 2023-10-16T08:39:23.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef09a9640 1 -- 172.21.15.40:0/1328288731 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3ed8003530 con 0x7f3eec0d00f0 2023-10-16T08:39:23.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef09a9640 1 -- 172.21.15.40:0/1328288731 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4171211262 0 0) 0x7f3ed8003530 con 0x7f3eec0d00f0 2023-10-16T08:39:23.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef09a9640 1 -- 172.21.15.40:0/1328288731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3eec0afb90 msgr2=0x7f3eec0cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:23.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef09a9640 1 --2- 172.21.15.40:0/1328288731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3eec0afb90 0x7f3eec0cfbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef09a9640 1 -- 172.21.15.40:0/1328288731 >> v1:172.21.15.86:6789/0 conn(0x7f3eec0ce540 legacy=0x7f3eec0cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:23.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef09a9640 1 -- 172.21.15.40:0/1328288731 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec0cc6d0 con 0x7f3eec0d00f0 2023-10-16T08:39:23.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef09a9640 1 -- 172.21.15.40:0/1328288731 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f3ee4001a00 con 0x7f3eec0d00f0 2023-10-16T08:39:23.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef09a9640 1 -- 172.21.15.40:0/1328288731 >> v1:172.21.15.40:6789/0 conn(0x7f3eec0d00f0 legacy=0x7f3eec164750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:23.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef09a9640 1 --2- 172.21.15.40:0/1328288731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3eec0afb90 0x7f3ed8003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:23.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef09a9640 1 --2- 172.21.15.40:0/1328288731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ed8003f60 0x7f3ed80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:23.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef09a9640 1 --2- 172.21.15.40:0/1328288731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ed8004d10 0x7f3ed8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:23.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef09a9640 1 -- 172.21.15.40:0/1328288731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec0cde80 con 0x7f3ed8003f60 2023-10-16T08:39:23.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef09a9640 1 -- 172.21.15.40:0/1328288731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed800bad0 con 0x7f3ed8004d10 2023-10-16T08:39:23.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef09a9640 1 -- 172.21.15.40:0/1328288731 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ed800bc90 con 0x7f3eec0afb90 2023-10-16T08:39:23.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef11aa640 1 --2- 172.21.15.40:0/1328288731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ed8003f60 0x7f3ed80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:23.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef21ac640 1 --2- 172.21.15.40:0/1328288731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ed8004d10 0x7f3ed8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:23.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef19ab640 1 --2- 172.21.15.40:0/1328288731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3eec0afb90 0x7f3ed8003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:23.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef11aa640 1 -- 172.21.15.40:0/1328288731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3eec0afb90 msgr2=0x7f3ed8003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:23.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef11aa640 1 --2- 172.21.15.40:0/1328288731 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3eec0afb90 0x7f3ed8003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef11aa640 1 -- 172.21.15.40:0/1328288731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ed8004d10 msgr2=0x7f3ed8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:23.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef11aa640 1 --2- 172.21.15.40:0/1328288731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ed8004d10 0x7f3ed8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef11aa640 1 -- 172.21.15.40:0/1328288731 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ed8003530 con 0x7f3ed8003f60 2023-10-16T08:39:23.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.799+0000 7f3ef21ac640 1 --2- 172.21.15.40:0/1328288731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ed8004d10 0x7f3ed8005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:23.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.803+0000 7f3ef11aa640 1 --2- 172.21.15.40:0/1328288731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ed8003f60 0x7f3ed80047d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3edc009ae0 tx=0x7f3edc01c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2702bd252d954702 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:23.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.803+0000 7f3ef09a9640 1 -- 172.21.15.40:0/1328288731 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc0a5580 con 0x7f3ed8003f60 2023-10-16T08:39:23.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.803+0000 7f3ef09a9640 1 -- 172.21.15.40:0/1328288731 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3edc0a56e0 con 0x7f3ed8003f60 2023-10-16T08:39:23.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.803+0000 7f3ef09a9640 1 -- 172.21.15.40:0/1328288731 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc0a5a90 con 0x7f3ed8003f60 2023-10-16T08:39:23.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.803+0000 7f3ef3c0f640 1 -- 172.21.15.40:0/1328288731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ed8003f60 msgr2=0x7f3ed80047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:23.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.803+0000 7f3ef3c0f640 1 --2- 172.21.15.40:0/1328288731 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3ed8003f60 0x7f3ed80047d0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f3edc009ae0 tx=0x7f3edc01c990 comp rx=0 tx=0).stop 2023-10-16T08:39:23.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.803+0000 7f3ef21ac640 1 -- 172.21.15.40:0/1328288731 reap_dead start 2023-10-16T08:39:23.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.803+0000 7f3ef3c0f640 1 -- 172.21.15.40:0/1328288731 shutdown_connections 2023-10-16T08:39:23.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.803+0000 7f3ef3c0f640 1 -- 172.21.15.40:0/1328288731 >> 172.21.15.40:0/1328288731 conn(0x7f3eec065e80 msgr2=0x7f3eec066290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:23.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.803+0000 7f3ef3c0f640 1 -- 172.21.15.40:0/1328288731 shutdown_connections 2023-10-16T08:39:23.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.803+0000 7f3ef3c0f640 1 -- 172.21.15.40:0/1328288731 wait complete. 2023-10-16T08:39:23.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.803+0000 7f3ef3c0f640 1 Processor -- start 2023-10-16T08:39:23.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.803+0000 7f3ef3c0f640 1 -- start start 2023-10-16T08:39:23.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef3c0f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ed8003f60 0x7f3eec148df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:23.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef3c0f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eec0b5a20 0x7f3eec1e35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:23.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef3c0f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eec1e3b30 0x7f3eec0c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:23.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef3c0f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec1e3fb0 con 0x7f3eec0b5a20 2023-10-16T08:39:23.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef3c0f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec0b5590 con 0x7f3eec1e3b30 2023-10-16T08:39:23.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef3c0f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eec0b4980 con 0x7f3ed8003f60 2023-10-16T08:39:23.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef11aa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eec0b5a20 0x7f3eec1e35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:23.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef11aa640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eec0b5a20 0x7f3eec1e35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45106/0 (socket says 172.21.15.40:45106) 2023-10-16T08:39:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef11aa640 1 -- 172.21.15.40:0/2948309523 learned_addr learned my addr 172.21.15.40:0/2948309523 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef21ac640 1 --2- 172.21.15.40:0/2948309523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eec1e3b30 0x7f3eec0c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef19ab640 1 --2- 172.21.15.40:0/2948309523 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ed8003f60 0x7f3eec148df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:23.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef11aa640 1 -- 172.21.15.40:0/2948309523 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ed8003f60 msgr2=0x7f3eec148df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:23.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef11aa640 1 --2- 172.21.15.40:0/2948309523 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ed8003f60 0x7f3eec148df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef11aa640 1 -- 172.21.15.40:0/2948309523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eec1e3b30 msgr2=0x7f3eec0c4e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:23.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef11aa640 1 --2- 172.21.15.40:0/2948309523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eec1e3b30 0x7f3eec0c4e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef11aa640 1 -- 172.21.15.40:0/2948309523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eec0b4c60 con 0x7f3eec0b5a20 2023-10-16T08:39:23.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef21ac640 1 --2- 172.21.15.40:0/2948309523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eec1e3b30 0x7f3eec0c4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:23.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef11aa640 1 --2- 172.21.15.40:0/2948309523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eec0b5a20 0x7f3eec1e35f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3edc01c7c0 tx=0x7f3ed8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:23.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ee27fc640 1 -- 172.21.15.40:0/2948309523 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc0bc070 con 0x7f3eec0b5a20 2023-10-16T08:39:23.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ee27fc640 1 -- 172.21.15.40:0/2948309523 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3edc004020 con 0x7f3eec0b5a20 2023-10-16T08:39:23.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef3c0f640 1 -- 172.21.15.40:0/2948309523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eec0b4f40 con 0x7f3eec0b5a20 2023-10-16T08:39:23.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ee27fc640 1 -- 172.21.15.40:0/2948309523 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3edc0a5580 con 0x7f3eec0b5a20 2023-10-16T08:39:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef3c0f640 1 -- 172.21.15.40:0/2948309523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eec149330 con 0x7f3eec0b5a20 2023-10-16T08:39:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ef3c0f640 1 -- 172.21.15.40:0/2948309523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eec04e9c0 con 0x7f3eec0b5a20 2023-10-16T08:39:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ee27fc640 1 -- 172.21.15.40:0/2948309523 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3edc0b4040 con 0x7f3eec0b5a20 2023-10-16T08:39:23.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ee27fc640 1 --2- 172.21.15.40:0/2948309523 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3ed00664a0 0x7f3ed0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.807+0000 7f3ee27fc640 1 -- 172.21.15.40:0/2948309523 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3edc11d950 con 0x7f3eec0b5a20 2023-10-16T08:39:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.811+0000 7f3ef19ab640 1 --2- 172.21.15.40:0/2948309523 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3ed00664a0 0x7f3ed0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:23.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.811+0000 7f3ef19ab640 1 --2- 172.21.15.40:0/2948309523 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3ed00664a0 0x7f3ed0068960 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8007ad0 tx=0x7f3ee800a8a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:23.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.815+0000 7f3ee27fc640 1 -- 172.21.15.40:0/2948309523 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f3edc0eaf80 con 0x7f3eec0b5a20 2023-10-16T08:39:23.970 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.963+0000 7f3ef3c0f640 1 -- 172.21.15.40:0/2948309523 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3eec0b56d0 con 0x7f3eec0b5a20 2023-10-16T08:39:23.971 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.967+0000 7f3ee27fc640 1 -- 172.21.15.40:0/2948309523 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3edc0ee5a0 con 0x7f3eec0b5a20 2023-10-16T08:39:23.972 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-16T08:39:23.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.967+0000 7f3ef3c0f640 1 -- 172.21.15.40:0/2948309523 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3ed00664a0 msgr2=0x7f3ed0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:23.973 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.967+0000 7f3ef3c0f640 1 --2- 172.21.15.40:0/2948309523 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3ed00664a0 0x7f3ed0068960 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f3ee8007ad0 tx=0x7f3ee800a8a0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.967+0000 7f3ef3c0f640 1 -- 172.21.15.40:0/2948309523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eec0b5a20 msgr2=0x7f3eec1e35f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:23.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.967+0000 7f3ef3c0f640 1 --2- 172.21.15.40:0/2948309523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eec0b5a20 0x7f3eec1e35f0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f3edc01c7c0 tx=0x7f3ed8005800 comp rx=0 tx=0).stop 2023-10-16T08:39:23.974 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.967+0000 7f3ef3c0f640 1 -- 172.21.15.40:0/2948309523 shutdown_connections 2023-10-16T08:39:23.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.967+0000 7f3ef3c0f640 1 --2- 172.21.15.40:0/2948309523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eec1e3b30 0x7f3eec0c4e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.967+0000 7f3ef3c0f640 1 --2- 172.21.15.40:0/2948309523 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3ed00664a0 0x7f3ed0068960 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.967+0000 7f3ef3c0f640 1 --2- 172.21.15.40:0/2948309523 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3eec0b5a20 0x7f3eec1e35f0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.967+0000 7f3ef3c0f640 1 --2- 172.21.15.40:0/2948309523 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3ed8003f60 0x7f3eec148df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:23.975 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.967+0000 7f3ef3c0f640 1 -- 172.21.15.40:0/2948309523 >> 172.21.15.40:0/2948309523 conn(0x7f3eec065e80 msgr2=0x7f3eec066290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:23.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.967+0000 7f3ef3c0f640 1 -- 172.21.15.40:0/2948309523 shutdown_connections 2023-10-16T08:39:23.976 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:23.967+0000 7f3ef3c0f640 1 -- 172.21.15.40:0/2948309523 wait complete. 2023-10-16T08:39:23.989 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:39:24.108 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.103+0000 7fbf8a8a4640 1 Processor -- start 2023-10-16T08:39:24.109 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.103+0000 7fbf8a8a4640 1 -- start start 2023-10-16T08:39:24.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.103+0000 7fbf8a8a4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbf840cae50 0x7fbf8406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.110 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.103+0000 7fbf8a8a4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf840adca0 con 0x7fbf840adec0 2023-10-16T08:39:24.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.103+0000 7fbf8a8a4640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf840c5840 con 0x7fbf8415b040 2023-10-16T08:39:24.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.103+0000 7fbf8a8a4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf840cf890 con 0x7fbf840cae50 2023-10-16T08:39:24.111 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.103+0000 7fbf837fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbf840cae50 0x7fbf8406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.103+0000 7fbf837fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbf840cae50 0x7fbf8406b7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45116/0 (socket says 172.21.15.40:45116) 2023-10-16T08:39:24.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.103+0000 7fbf88e41640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbf840adec0 0x7fbf8406bcf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32792/0 (socket says 172.21.15.40:32792) 2023-10-16T08:39:24.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.103+0000 7fbf837fe640 1 -- 172.21.15.40:0/2945812439 learned_addr learned my addr 172.21.15.40:0/2945812439 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:24.112 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.103+0000 7fbf837fe640 1 -- 172.21.15.40:0/2945812439 >> v1:172.21.15.86:6789/0 conn(0x7fbf8415b040 legacy=0x7fbf840ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:24.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.103+0000 7fbf82ffd640 1 -- 172.21.15.40:0/2945812439 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791808241 0 0) 0x7fbf840adca0 con 0x7fbf840adec0 2023-10-16T08:39:24.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.103+0000 7fbf837fe640 1 -- 172.21.15.40:0/2945812439 >> v1:172.21.15.40:6789/0 conn(0x7fbf840adec0 legacy=0x7fbf8406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:24.113 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.103+0000 7fbf837fe640 1 -- 172.21.15.40:0/2945812439 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf840cf5b0 con 0x7fbf840cae50 2023-10-16T08:39:24.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf837fe640 1 --2- 172.21.15.40:0/2945812439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf840cae50 0x7fbf8406b7b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009e50 tx=0x7fbf7801c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efb9c0cbd95de701 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:24.117 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf82ffd640 1 -- 172.21.15.40:0/2945812439 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf780a5690 con 0x7fbf840cae50 2023-10-16T08:39:24.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf82ffd640 1 -- 172.21.15.40:0/2945812439 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf780a57f0 con 0x7fbf840cae50 2023-10-16T08:39:24.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf82ffd640 1 -- 172.21.15.40:0/2945812439 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf780a5b80 con 0x7fbf840cae50 2023-10-16T08:39:24.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf8a8a4640 1 -- 172.21.15.40:0/2945812439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf840cae50 msgr2=0x7fbf8406b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:24.118 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf8a8a4640 1 --2- 172.21.15.40:0/2945812439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf840cae50 0x7fbf8406b7b0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbf78009e50 tx=0x7fbf7801c8c0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf8a8a4640 1 -- 172.21.15.40:0/2945812439 shutdown_connections 2023-10-16T08:39:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf8a8a4640 1 --2- 172.21.15.40:0/2945812439 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf840cae50 0x7fbf8406b7b0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf8a8a4640 1 -- 172.21.15.40:0/2945812439 >> 172.21.15.40:0/2945812439 conn(0x7fbf840ce960 msgr2=0x7fbf840ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:24.119 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf8a8a4640 1 -- 172.21.15.40:0/2945812439 shutdown_connections 2023-10-16T08:39:24.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf8a8a4640 1 -- 172.21.15.40:0/2945812439 wait complete. 2023-10-16T08:39:24.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf8a8a4640 1 Processor -- start 2023-10-16T08:39:24.120 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf8a8a4640 1 -- start start 2023-10-16T08:39:24.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf8a8a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf840adec0 0x7fbf84153330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf8a8a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf840cae50 0x7fbf8415dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf8a8a4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf8415b040 0x7fbf840c2490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.121 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf8a8a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8414bbf0 con 0x7fbf840cae50 2023-10-16T08:39:24.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf8a8a4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8414b000 con 0x7fbf840adec0 2023-10-16T08:39:24.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf8a8a4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf8414b2c0 con 0x7fbf8415b040 2023-10-16T08:39:24.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf837fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf840cae50 0x7fbf8415dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.122 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf837fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf840cae50 0x7fbf8415dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45128/0 (socket says 172.21.15.40:45128) 2023-10-16T08:39:24.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf837fe640 1 -- 172.21.15.40:0/1734081042 learned_addr learned my addr 172.21.15.40:0/1734081042 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:24.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf83fff640 1 --2- 172.21.15.40:0/1734081042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf840adec0 0x7fbf84153330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.123 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf88e41640 1 --2- 172.21.15.40:0/1734081042 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf8415b040 0x7fbf840c2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf837fe640 1 -- 172.21.15.40:0/1734081042 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf8415b040 msgr2=0x7fbf840c2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf837fe640 1 --2- 172.21.15.40:0/1734081042 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf8415b040 0x7fbf840c2490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf837fe640 1 -- 172.21.15.40:0/1734081042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf840adec0 msgr2=0x7fbf84153330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:24.124 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf837fe640 1 --2- 172.21.15.40:0/1734081042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf840adec0 0x7fbf84153330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf837fe640 1 -- 172.21.15.40:0/1734081042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf78009b30 con 0x7fbf840cae50 2023-10-16T08:39:24.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.107+0000 7fbf83fff640 1 --2- 172.21.15.40:0/1734081042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf840adec0 0x7fbf84153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:24.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.111+0000 7fbf837fe640 1 --2- 172.21.15.40:0/1734081042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf840cae50 0x7fbf8415dbc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbf780af040 tx=0x7fbf8406cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:24.125 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.111+0000 7fbf80ff9640 1 -- 172.21.15.40:0/1734081042 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf780b8670 con 0x7fbf840cae50 2023-10-16T08:39:24.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.111+0000 7fbf80ff9640 1 -- 172.21.15.40:0/1734081042 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf780b87f0 con 0x7fbf840cae50 2023-10-16T08:39:24.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.111+0000 7fbf8a8a4640 1 -- 172.21.15.40:0/1734081042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf8414b5a0 con 0x7fbf840cae50 2023-10-16T08:39:24.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.111+0000 7fbf80ff9640 1 -- 172.21.15.40:0/1734081042 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf780b8ba0 con 0x7fbf840cae50 2023-10-16T08:39:24.126 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.111+0000 7fbf8a8a4640 1 -- 172.21.15.40:0/1734081042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf84153870 con 0x7fbf840cae50 2023-10-16T08:39:24.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.111+0000 7fbf8a8a4640 1 -- 172.21.15.40:0/1734081042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf8414bd60 con 0x7fbf840cae50 2023-10-16T08:39:24.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.111+0000 7fbf80ff9640 1 -- 172.21.15.40:0/1734081042 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbf780b8d00 con 0x7fbf840cae50 2023-10-16T08:39:24.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.111+0000 7fbf80ff9640 1 --2- 172.21.15.40:0/1734081042 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbf700664a0 0x7fbf70068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.127 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.111+0000 7fbf80ff9640 1 -- 172.21.15.40:0/1734081042 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf7811f400 con 0x7fbf840cae50 2023-10-16T08:39:24.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.111+0000 7fbf83fff640 1 --2- 172.21.15.40:0/1734081042 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbf700664a0 0x7fbf70068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.128 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.111+0000 7fbf83fff640 1 --2- 172.21.15.40:0/1734081042 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbf700664a0 0x7fbf70068960 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbf74000f70 tx=0x7fbf74009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:24.129 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.115+0000 7fbf80ff9640 1 -- 172.21.15.40:0/1734081042 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fbf780ec9a0 con 0x7fbf840cae50 2023-10-16T08:39:24.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.283+0000 7fbf8a8a4640 1 -- 172.21.15.40:0/1734081042 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fbf8404e9c0 con 0x7fbf840cae50 2023-10-16T08:39:24.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.283+0000 7fbf80ff9640 1 -- 172.21.15.40:0/1734081042 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7fbf780ace80 con 0x7fbf840cae50 2023-10-16T08:39:24.291 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-16T08:39:24.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.287+0000 7fbf8a8a4640 1 -- 172.21.15.40:0/1734081042 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbf700664a0 msgr2=0x7fbf70068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:24.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.287+0000 7fbf8a8a4640 1 --2- 172.21.15.40:0/1734081042 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbf700664a0 0x7fbf70068960 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fbf74000f70 tx=0x7fbf74009040 comp rx=0 tx=0).stop 2023-10-16T08:39:24.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.287+0000 7fbf8a8a4640 1 -- 172.21.15.40:0/1734081042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf840cae50 msgr2=0x7fbf8415dbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:24.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.287+0000 7fbf8a8a4640 1 --2- 172.21.15.40:0/1734081042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf840cae50 0x7fbf8415dbc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fbf780af040 tx=0x7fbf8406cc10 comp rx=0 tx=0).stop 2023-10-16T08:39:24.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.287+0000 7fbf8a8a4640 1 -- 172.21.15.40:0/1734081042 shutdown_connections 2023-10-16T08:39:24.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.287+0000 7fbf8a8a4640 1 --2- 172.21.15.40:0/1734081042 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbf8415b040 0x7fbf840c2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.287+0000 7fbf8a8a4640 1 --2- 172.21.15.40:0/1734081042 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbf700664a0 0x7fbf70068960 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.287+0000 7fbf8a8a4640 1 --2- 172.21.15.40:0/1734081042 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbf840cae50 0x7fbf8415dbc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.287+0000 7fbf8a8a4640 1 --2- 172.21.15.40:0/1734081042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf840adec0 0x7fbf84153330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.287+0000 7fbf8a8a4640 1 -- 172.21.15.40:0/1734081042 >> 172.21.15.40:0/1734081042 conn(0x7fbf840ce960 msgr2=0x7fbf84154910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:24.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.287+0000 7fbf8a8a4640 1 -- 172.21.15.40:0/1734081042 shutdown_connections 2023-10-16T08:39:24.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.287+0000 7fbf8a8a4640 1 -- 172.21.15.40:0/1734081042 wait complete. 2023-10-16T08:39:24.308 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:39:24.309 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:39:24.315 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:39:24.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c6c712640 1 Processor -- start 2023-10-16T08:39:24.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c6c712640 1 -- start start 2023-10-16T08:39:24.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c6c712640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f3c640cc530 0x7f3c6406c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c6c712640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c640c8fb0 0x7f3c640c9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c6c712640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c640cd640 con 0x7f3c6415ca80 2023-10-16T08:39:24.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c6c712640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c640c99a0 con 0x7f3c640cc530 2023-10-16T08:39:24.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c6c712640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6406fbf0 con 0x7f3c640c8fb0 2023-10-16T08:39:24.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c6a4ae640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f3c6415ca80 0x7f3c6406f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:32798/0 (socket says 172.21.15.40:32798) 2023-10-16T08:39:24.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c6a4ae640 1 -- 172.21.15.40:0/1082257048 learned_addr learned my addr 172.21.15.40:0/1082257048 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:24.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c694ac640 1 -- 172.21.15.40:0/1082257048 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2595822238 0 0) 0x7f3c640cd640 con 0x7f3c6415ca80 2023-10-16T08:39:24.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c69cad640 1 --2- 172.21.15.40:0/1082257048 >> v2:172.21.15.86:3300/0 conn(0x7f3c640cc530 0x7f3c6406c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c694ac640 1 -- 172.21.15.40:0/1082257048 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3c50003450 con 0x7f3c6415ca80 2023-10-16T08:39:24.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c6acaf640 1 --2- 172.21.15.40:0/1082257048 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c640c8fb0 0x7f3c640c9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c694ac640 1 -- 172.21.15.40:0/1082257048 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2719801601 0 0) 0x7f3c50003450 con 0x7f3c6415ca80 2023-10-16T08:39:24.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c694ac640 1 -- 172.21.15.40:0/1082257048 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c640c8fb0 msgr2=0x7f3c640c9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:24.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c694ac640 1 --2- 172.21.15.40:0/1082257048 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c640c8fb0 0x7f3c640c9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c694ac640 1 -- 172.21.15.40:0/1082257048 >> v2:172.21.15.86:3300/0 conn(0x7f3c640cc530 msgr2=0x7f3c6406c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:24.442 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c694ac640 1 --2- 172.21.15.40:0/1082257048 >> v2:172.21.15.86:3300/0 conn(0x7f3c640cc530 0x7f3c6406c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c694ac640 1 -- 172.21.15.40:0/1082257048 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c640cca30 con 0x7f3c6415ca80 2023-10-16T08:39:24.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c694ac640 1 -- 172.21.15.40:0/1082257048 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f3c54001680 con 0x7f3c6415ca80 2023-10-16T08:39:24.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c694ac640 1 -- 172.21.15.40:0/1082257048 >> v1:172.21.15.40:6789/0 conn(0x7f3c6415ca80 legacy=0x7f3c6406f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:24.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c694ac640 1 --2- 172.21.15.40:0/1082257048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c50003ae0 0x7f3c50003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c694ac640 1 --2- 172.21.15.40:0/1082257048 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c640c8fb0 0x7f3c5000a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c694ac640 1 --2- 172.21.15.40:0/1082257048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c5000af10 0x7f3c5000d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c694ac640 1 -- 172.21.15.40:0/1082257048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c640c99a0 con 0x7f3c50003ae0 2023-10-16T08:39:24.444 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c694ac640 1 -- 172.21.15.40:0/1082257048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6406fbf0 con 0x7f3c5000af10 2023-10-16T08:39:24.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c694ac640 1 -- 172.21.15.40:0/1082257048 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c5000dc00 con 0x7f3c640c8fb0 2023-10-16T08:39:24.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c6a4ae640 1 --2- 172.21.15.40:0/1082257048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c50003ae0 0x7f3c50003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c6a4ae640 1 -- 172.21.15.40:0/1082257048 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c640c8fb0 msgr2=0x7f3c5000a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:24.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c69cad640 1 --2- 172.21.15.40:0/1082257048 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c640c8fb0 0x7f3c5000a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c6a4ae640 1 --2- 172.21.15.40:0/1082257048 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c640c8fb0 0x7f3c5000a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c6acaf640 1 --2- 172.21.15.40:0/1082257048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c5000af10 0x7f3c5000d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c6a4ae640 1 -- 172.21.15.40:0/1082257048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c5000af10 msgr2=0x7f3c5000d3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:24.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c6a4ae640 1 --2- 172.21.15.40:0/1082257048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c5000af10 0x7f3c5000d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.431+0000 7f3c6a4ae640 1 -- 172.21.15.40:0/1082257048 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c50003450 con 0x7f3c50003ae0 2023-10-16T08:39:24.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6a4ae640 1 --2- 172.21.15.40:0/1082257048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c50003ae0 0x7f3c50003f00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400c290 tx=0x7f3c5401ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea76fde8aa6db27 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:24.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c694ac640 1 -- 172.21.15.40:0/1082257048 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c540058e0 con 0x7f3c50003ae0 2023-10-16T08:39:24.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c694ac640 1 -- 172.21.15.40:0/1082257048 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c54005a40 con 0x7f3c50003ae0 2023-10-16T08:39:24.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c694ac640 1 -- 172.21.15.40:0/1082257048 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54005df0 con 0x7f3c50003ae0 2023-10-16T08:39:24.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6c712640 1 -- 172.21.15.40:0/1082257048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c50003ae0 msgr2=0x7f3c50003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:24.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6c712640 1 --2- 172.21.15.40:0/1082257048 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c50003ae0 0x7f3c50003f00 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400c290 tx=0x7f3c5401ccb0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6acaf640 1 -- 172.21.15.40:0/1082257048 reap_dead start 2023-10-16T08:39:24.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6c712640 1 -- 172.21.15.40:0/1082257048 shutdown_connections 2023-10-16T08:39:24.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6c712640 1 -- 172.21.15.40:0/1082257048 >> 172.21.15.40:0/1082257048 conn(0x7f3c640ce960 msgr2=0x7f3c640ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:24.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6c712640 1 -- 172.21.15.40:0/1082257048 shutdown_connections 2023-10-16T08:39:24.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6c712640 1 -- 172.21.15.40:0/1082257048 wait complete. 2023-10-16T08:39:24.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6c712640 1 Processor -- start 2023-10-16T08:39:24.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6c712640 1 -- start start 2023-10-16T08:39:24.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6c712640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c50003ae0 0x7f3c64149f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6c712640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c64161890 0x7f3c6414b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6c712640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c640aedb0 0x7f3c640af230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6c712640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c640af770 con 0x7f3c50003ae0 2023-10-16T08:39:24.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6c712640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6414a470 con 0x7f3c640aedb0 2023-10-16T08:39:24.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6c712640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c6414bc10 con 0x7f3c64161890 2023-10-16T08:39:24.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6a4ae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c50003ae0 0x7f3c64149f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6a4ae640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c50003ae0 0x7f3c64149f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:45152/0 (socket says 172.21.15.40:45152) 2023-10-16T08:39:24.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6a4ae640 1 -- 172.21.15.40:0/2020083963 learned_addr learned my addr 172.21.15.40:0/2020083963 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:24.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c69cad640 1 --2- 172.21.15.40:0/2020083963 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c64161890 0x7f3c6414b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6acaf640 1 --2- 172.21.15.40:0/2020083963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c640aedb0 0x7f3c640af230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6a4ae640 1 -- 172.21.15.40:0/2020083963 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c64161890 msgr2=0x7f3c6414b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:24.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6a4ae640 1 --2- 172.21.15.40:0/2020083963 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c64161890 0x7f3c6414b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6a4ae640 1 -- 172.21.15.40:0/2020083963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c640aedb0 msgr2=0x7f3c640af230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:24.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.435+0000 7f3c6a4ae640 1 --2- 172.21.15.40:0/2020083963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c640aedb0 0x7f3c640af230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.439+0000 7f3c6a4ae640 1 -- 172.21.15.40:0/2020083963 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c6414bef0 con 0x7f3c50003ae0 2023-10-16T08:39:24.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.439+0000 7f3c6acaf640 1 --2- 172.21.15.40:0/2020083963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c640aedb0 0x7f3c640af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:24.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.439+0000 7f3c69cad640 1 --2- 172.21.15.40:0/2020083963 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c64161890 0x7f3c6414b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:24.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.439+0000 7f3c6a4ae640 1 --2- 172.21.15.40:0/2020083963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c50003ae0 0x7f3c64149f30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400c290 tx=0x7f3c50003700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:24.459 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.439+0000 7f3c5affd640 1 -- 172.21.15.40:0/2020083963 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54001c90 con 0x7f3c50003ae0 2023-10-16T08:39:24.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.439+0000 7f3c5affd640 1 -- 172.21.15.40:0/2020083963 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3c54005dd0 con 0x7f3c50003ae0 2023-10-16T08:39:24.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.439+0000 7f3c6c712640 1 -- 172.21.15.40:0/2020083963 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c6414c1d0 con 0x7f3c50003ae0 2023-10-16T08:39:24.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.439+0000 7f3c5affd640 1 -- 172.21.15.40:0/2020083963 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3c54001660 con 0x7f3c50003ae0 2023-10-16T08:39:24.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.439+0000 7f3c6c712640 1 -- 172.21.15.40:0/2020083963 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c6414b840 con 0x7f3c50003ae0 2023-10-16T08:39:24.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.439+0000 7f3c6c712640 1 -- 172.21.15.40:0/2020083963 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c6404e9c0 con 0x7f3c50003ae0 2023-10-16T08:39:24.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.439+0000 7f3c5affd640 1 -- 172.21.15.40:0/2020083963 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f3c540a5790 con 0x7f3c50003ae0 2023-10-16T08:39:24.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.439+0000 7f3c5affd640 1 --2- 172.21.15.40:0/2020083963 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3c480664d0 0x7f3c48068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.439+0000 7f3c5affd640 1 -- 172.21.15.40:0/2020083963 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3c5411d720 con 0x7f3c50003ae0 2023-10-16T08:39:24.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.439+0000 7f3c69cad640 1 --2- 172.21.15.40:0/2020083963 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3c480664d0 0x7f3c48068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.439+0000 7f3c69cad640 1 --2- 172.21.15.40:0/2020083963 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3c480664d0 0x7f3c48068990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3c6415be00 tx=0x7f3c60008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:24.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.443+0000 7f3c5affd640 1 -- 172.21.15.40:0/2020083963 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f3c540eab00 con 0x7f3c50003ae0 2023-10-16T08:39:24.600 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.595+0000 7f3c6c712640 1 -- 172.21.15.40:0/2020083963 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3c640b48e0 con 0x7f3c50003ae0 2023-10-16T08:39:24.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.595+0000 7f3c5affd640 1 -- 172.21.15.40:0/2020083963 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16445 (secure 0 0 0) 0x7f3c540ee120 con 0x7f3c50003ae0 2023-10-16T08:39:24.602 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:39:24.603 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":20,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:38:46.611098+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:39:24.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.599+0000 7f3c6c712640 1 -- 172.21.15.40:0/2020083963 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3c480664d0 msgr2=0x7f3c48068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:24.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.599+0000 7f3c6c712640 1 --2- 172.21.15.40:0/2020083963 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3c480664d0 0x7f3c48068990 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f3c6415be00 tx=0x7f3c60008040 comp rx=0 tx=0).stop 2023-10-16T08:39:24.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.599+0000 7f3c6c712640 1 -- 172.21.15.40:0/2020083963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c50003ae0 msgr2=0x7f3c64149f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:24.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.599+0000 7f3c6c712640 1 --2- 172.21.15.40:0/2020083963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c50003ae0 0x7f3c64149f30 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f3c5400c290 tx=0x7f3c50003700 comp rx=0 tx=0).stop 2023-10-16T08:39:24.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.599+0000 7f3c6c712640 1 -- 172.21.15.40:0/2020083963 shutdown_connections 2023-10-16T08:39:24.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.599+0000 7f3c6c712640 1 --2- 172.21.15.40:0/2020083963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c640aedb0 0x7f3c640af230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.599+0000 7f3c6c712640 1 --2- 172.21.15.40:0/2020083963 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f3c64161890 0x7f3c6414b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.599+0000 7f3c6c712640 1 --2- 172.21.15.40:0/2020083963 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f3c480664d0 0x7f3c48068990 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.599+0000 7f3c6c712640 1 --2- 172.21.15.40:0/2020083963 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f3c50003ae0 0x7f3c64149f30 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.608 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.599+0000 7f3c6c712640 1 -- 172.21.15.40:0/2020083963 >> 172.21.15.40:0/2020083963 conn(0x7f3c640ce960 msgr2=0x7f3c640afd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:24.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.599+0000 7f3c6c712640 1 -- 172.21.15.40:0/2020083963 shutdown_connections 2023-10-16T08:39:24.609 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.599+0000 7f3c6c712640 1 -- 172.21.15.40:0/2020083963 wait complete. 2023-10-16T08:39:24.620 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:38:33.014668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:38:43.055215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:38:43.971666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:39:24.621 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:39:24.746 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.739+0000 7ffb27c2d640 1 Processor -- start 2023-10-16T08:39:24.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.739+0000 7ffb27c2d640 1 -- start start 2023-10-16T08:39:24.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb27c2d640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ffb200c9f00 0x7ffb200cc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb27c2d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb2015b490 con 0x7ffb20159dd0 2023-10-16T08:39:24.750 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb27c2d640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb2006d230 con 0x7ffb200adec0 2023-10-16T08:39:24.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb27c2d640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb2006cd70 con 0x7ffb200c9f00 2023-10-16T08:39:24.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb259c9640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ffb20159dd0 0x7ffb200c8f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36968/0 (socket says 172.21.15.40:36968) 2023-10-16T08:39:24.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb259c9640 1 -- 172.21.15.40:0/1795757543 learned_addr learned my addr 172.21.15.40:0/1795757543 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:24.751 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb261ca640 1 --2- 172.21.15.40:0/1795757543 >> v2:172.21.15.86:3300/0 conn(0x7ffb200c9f00 0x7ffb200cc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb249c7640 1 -- 172.21.15.40:0/1795757543 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3199045090 0 0) 0x7ffb2015b490 con 0x7ffb20159dd0 2023-10-16T08:39:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb249c7640 1 -- 172.21.15.40:0/1795757543 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ffb0c003530 con 0x7ffb20159dd0 2023-10-16T08:39:24.752 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb249c7640 1 -- 172.21.15.40:0/1795757543 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 854783703 0 0) 0x7ffb0c003530 con 0x7ffb20159dd0 2023-10-16T08:39:24.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb249c7640 1 -- 172.21.15.40:0/1795757543 >> v2:172.21.15.86:3300/0 conn(0x7ffb200c9f00 msgr2=0x7ffb200cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:24.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb249c7640 1 --2- 172.21.15.40:0/1795757543 >> v2:172.21.15.86:3300/0 conn(0x7ffb200c9f00 0x7ffb200cc460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb249c7640 1 -- 172.21.15.40:0/1795757543 >> v1:172.21.15.86:6789/0 conn(0x7ffb200adec0 legacy=0x7ffb200c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:24.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb249c7640 1 -- 172.21.15.40:0/1795757543 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb2006cf50 con 0x7ffb20159dd0 2023-10-16T08:39:24.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb249c7640 1 -- 172.21.15.40:0/1795757543 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7ffb18001a00 con 0x7ffb20159dd0 2023-10-16T08:39:24.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb249c7640 1 -- 172.21.15.40:0/1795757543 >> v1:172.21.15.40:6789/0 conn(0x7ffb20159dd0 legacy=0x7ffb200c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:24.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb249c7640 1 --2- 172.21.15.40:0/1795757543 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0c0039f0 0x7ffb0c003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb249c7640 1 --2- 172.21.15.40:0/1795757543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb0c0047d0 0x7ffb0c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.755 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb249c7640 1 --2- 172.21.15.40:0/1795757543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb0c005190 0x7ffb0c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb249c7640 1 -- 172.21.15.40:0/1795757543 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2006cd70 con 0x7ffb0c0047d0 2023-10-16T08:39:24.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb249c7640 1 -- 172.21.15.40:0/1795757543 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c00c0c0 con 0x7ffb0c005190 2023-10-16T08:39:24.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb249c7640 1 -- 172.21.15.40:0/1795757543 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb0c00c280 con 0x7ffb0c0039f0 2023-10-16T08:39:24.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb251c8640 1 --2- 172.21.15.40:0/1795757543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb0c0047d0 0x7ffb0c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb261ca640 1 --2- 172.21.15.40:0/1795757543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb0c005190 0x7ffb0c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb259c9640 1 --2- 172.21.15.40:0/1795757543 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0c0039f0 0x7ffb0c003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb251c8640 1 -- 172.21.15.40:0/1795757543 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0c0039f0 msgr2=0x7ffb0c003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:24.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb251c8640 1 --2- 172.21.15.40:0/1795757543 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0c0039f0 0x7ffb0c003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb251c8640 1 -- 172.21.15.40:0/1795757543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb0c005190 msgr2=0x7ffb0c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:24.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb251c8640 1 --2- 172.21.15.40:0/1795757543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb0c005190 0x7ffb0c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb251c8640 1 -- 172.21.15.40:0/1795757543 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb0c003530 con 0x7ffb0c0047d0 2023-10-16T08:39:24.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb259c9640 1 --2- 172.21.15.40:0/1795757543 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0c0039f0 0x7ffb0c003e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:24.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb261ca640 1 --2- 172.21.15.40:0/1795757543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb0c005190 0x7ffb0c00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:24.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.743+0000 7ffb261ca640 1 -- 172.21.15.40:0/1795757543 reap_dead start 2023-10-16T08:39:24.763 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb251c8640 1 --2- 172.21.15.40:0/1795757543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb0c0047d0 0x7ffb0c004c50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ffb10001750 tx=0x7ffb1001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bc350e6b8e7e29 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:24.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb249c7640 1 -- 172.21.15.40:0/1795757543 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb100052d0 con 0x7ffb0c0047d0 2023-10-16T08:39:24.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb249c7640 1 -- 172.21.15.40:0/1795757543 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb100a7050 con 0x7ffb0c0047d0 2023-10-16T08:39:24.764 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb249c7640 1 -- 172.21.15.40:0/1795757543 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb10003600 con 0x7ffb0c0047d0 2023-10-16T08:39:24.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb27c2d640 1 -- 172.21.15.40:0/1795757543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb0c0047d0 msgr2=0x7ffb0c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:24.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb27c2d640 1 --2- 172.21.15.40:0/1795757543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb0c0047d0 0x7ffb0c004c50 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7ffb10001750 tx=0x7ffb1001c3a0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb27c2d640 1 -- 172.21.15.40:0/1795757543 shutdown_connections 2023-10-16T08:39:24.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb27c2d640 1 --2- 172.21.15.40:0/1795757543 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb0c0047d0 0x7ffb0c004c50 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb27c2d640 1 -- 172.21.15.40:0/1795757543 >> 172.21.15.40:0/1795757543 conn(0x7ffb200ce890 msgr2=0x7ffb200ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:24.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb27c2d640 1 -- 172.21.15.40:0/1795757543 shutdown_connections 2023-10-16T08:39:24.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb27c2d640 1 -- 172.21.15.40:0/1795757543 wait complete. 2023-10-16T08:39:24.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb27c2d640 1 Processor -- start 2023-10-16T08:39:24.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb27c2d640 1 -- start start 2023-10-16T08:39:24.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb27c2d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0c0047d0 0x7ffb2015bc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb27c2d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2014fa30 0x7ffb2014ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb27c2d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb20150450 0x7ffb20151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb27c2d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb2014ffd0 con 0x7ffb20150450 2023-10-16T08:39:24.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb27c2d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb200c7fa0 con 0x7ffb2014fa30 2023-10-16T08:39:24.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb27c2d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb200c8280 con 0x7ffb0c0047d0 2023-10-16T08:39:24.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb261ca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb20150450 0x7ffb20151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb261ca640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb20150450 0x7ffb20151f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49744/0 (socket says 172.21.15.40:49744) 2023-10-16T08:39:24.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb261ca640 1 -- 172.21.15.40:0/3282635779 learned_addr learned my addr 172.21.15.40:0/3282635779 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:24.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb251c8640 1 --2- 172.21.15.40:0/3282635779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2014fa30 0x7ffb2014ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb261ca640 1 -- 172.21.15.40:0/3282635779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0c0047d0 msgr2=0x7ffb2015bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:24.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb259c9640 1 --2- 172.21.15.40:0/3282635779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0c0047d0 0x7ffb2015bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb261ca640 1 --2- 172.21.15.40:0/3282635779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0c0047d0 0x7ffb2015bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb261ca640 1 -- 172.21.15.40:0/3282635779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2014fa30 msgr2=0x7ffb2014ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:24.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb261ca640 1 --2- 172.21.15.40:0/3282635779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2014fa30 0x7ffb2014ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb261ca640 1 -- 172.21.15.40:0/3282635779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb200c8560 con 0x7ffb20150450 2023-10-16T08:39:24.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb261ca640 1 --2- 172.21.15.40:0/3282635779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb20150450 0x7ffb20151f30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ffb2006deb0 tx=0x7ffb1c010c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:24.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb167fc640 1 -- 172.21.15.40:0/3282635779 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c00e840 con 0x7ffb20150450 2023-10-16T08:39:24.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb167fc640 1 -- 172.21.15.40:0/3282635779 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb1c00e9a0 con 0x7ffb20150450 2023-10-16T08:39:24.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb27c2d640 1 -- 172.21.15.40:0/3282635779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb2015c170 con 0x7ffb20150450 2023-10-16T08:39:24.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb167fc640 1 -- 172.21.15.40:0/3282635779 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb1c00ed10 con 0x7ffb20150450 2023-10-16T08:39:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.747+0000 7ffb27c2d640 1 -- 172.21.15.40:0/3282635779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb201508d0 con 0x7ffb20150450 2023-10-16T08:39:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.751+0000 7ffb27c2d640 1 -- 172.21.15.40:0/3282635779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb20067a20 con 0x7ffb20150450 2023-10-16T08:39:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.751+0000 7ffb167fc640 1 -- 172.21.15.40:0/3282635779 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ffb1c0143f0 con 0x7ffb20150450 2023-10-16T08:39:24.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.751+0000 7ffb167fc640 1 --2- 172.21.15.40:0/3282635779 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ffb040664a0 0x7ffb04068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:24.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.751+0000 7ffb167fc640 1 -- 172.21.15.40:0/3282635779 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ffb1c084030 con 0x7ffb20150450 2023-10-16T08:39:24.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.751+0000 7ffb259c9640 1 --2- 172.21.15.40:0/3282635779 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ffb040664a0 0x7ffb04068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:24.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.751+0000 7ffb259c9640 1 --2- 172.21.15.40:0/3282635779 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ffb040664a0 0x7ffb04068960 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffb20068a60 tx=0x7ffb180012a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:24.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.755+0000 7ffb167fc640 1 -- 172.21.15.40:0/3282635779 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7ffb1c051540 con 0x7ffb20150450 2023-10-16T08:39:24.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.911+0000 7ffb27c2d640 1 -- 172.21.15.40:0/3282635779 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb200ce690 con 0x7ffb20150450 2023-10-16T08:39:24.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.911+0000 7ffb167fc640 1 -- 172.21.15.40:0/3282635779 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v20) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb1c054b60 con 0x7ffb20150450 2023-10-16T08:39:24.918 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-16T08:39:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.915+0000 7ffb27c2d640 1 -- 172.21.15.40:0/3282635779 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ffb040664a0 msgr2=0x7ffb04068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.915+0000 7ffb27c2d640 1 --2- 172.21.15.40:0/3282635779 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ffb040664a0 0x7ffb04068960 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7ffb20068a60 tx=0x7ffb180012a0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.915+0000 7ffb27c2d640 1 -- 172.21.15.40:0/3282635779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb20150450 msgr2=0x7ffb20151f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:24.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.915+0000 7ffb27c2d640 1 --2- 172.21.15.40:0/3282635779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb20150450 0x7ffb20151f30 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ffb2006deb0 tx=0x7ffb1c010c20 comp rx=0 tx=0).stop 2023-10-16T08:39:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.915+0000 7ffb27c2d640 1 -- 172.21.15.40:0/3282635779 shutdown_connections 2023-10-16T08:39:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.915+0000 7ffb27c2d640 1 --2- 172.21.15.40:0/3282635779 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ffb040664a0 0x7ffb04068960 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.915+0000 7ffb27c2d640 1 --2- 172.21.15.40:0/3282635779 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ffb20150450 0x7ffb20151f30 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.915+0000 7ffb27c2d640 1 --2- 172.21.15.40:0/3282635779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb2014fa30 0x7ffb2014ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.915+0000 7ffb27c2d640 1 --2- 172.21.15.40:0/3282635779 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ffb0c0047d0 0x7ffb2015bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:24.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.915+0000 7ffb27c2d640 1 -- 172.21.15.40:0/3282635779 >> 172.21.15.40:0/3282635779 conn(0x7ffb200ce890 msgr2=0x7ffb200c45d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:24.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.915+0000 7ffb27c2d640 1 -- 172.21.15.40:0/3282635779 shutdown_connections 2023-10-16T08:39:24.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:24.915+0000 7ffb27c2d640 1 -- 172.21.15.40:0/3282635779 wait complete. 2023-10-16T08:39:24.938 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:39:25.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe3012b3640 1 Processor -- start 2023-10-16T08:39:25.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe3012b3640 1 -- start start 2023-10-16T08:39:25.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe3012b3640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe2fc0acff0 0x7fe2fc06c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:25.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe3012b3640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2fc0add00 con 0x7fe2fc0cce70 2023-10-16T08:39:25.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe3012b3640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2fc0adfe0 con 0x7fe2fc15d020 2023-10-16T08:39:25.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe3012b3640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2fc06b3d0 con 0x7fe2fc0acff0 2023-10-16T08:39:25.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2fa7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe2fc0cce70 0x7fe2fc0ac8e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36984/0 (socket says 172.21.15.40:36984) 2023-10-16T08:39:25.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2fa7fc640 1 -- 172.21.15.40:0/194639657 learned_addr learned my addr 172.21.15.40:0/194639657 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:25.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2fb7fe640 1 --2- 172.21.15.40:0/194639657 >> v2:172.21.15.86:3300/0 conn(0x7fe2fc0acff0 0x7fe2fc06c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:25.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2f9ffb640 1 -- 172.21.15.40:0/194639657 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3580724830 0 0) 0x7fe2fc0add00 con 0x7fe2fc0cce70 2023-10-16T08:39:25.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2f9ffb640 1 -- 172.21.15.40:0/194639657 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2e4003530 con 0x7fe2fc0cce70 2023-10-16T08:39:25.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2f9ffb640 1 -- 172.21.15.40:0/194639657 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 230780079 0 0) 0x7fe2e4003530 con 0x7fe2fc0cce70 2023-10-16T08:39:25.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2f9ffb640 1 -- 172.21.15.40:0/194639657 >> v2:172.21.15.86:3300/0 conn(0x7fe2fc0acff0 msgr2=0x7fe2fc06c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:25.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2f9ffb640 1 --2- 172.21.15.40:0/194639657 >> v2:172.21.15.86:3300/0 conn(0x7fe2fc0acff0 0x7fe2fc06c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2f9ffb640 1 -- 172.21.15.40:0/194639657 >> v1:172.21.15.86:6789/0 conn(0x7fe2fc15d020 legacy=0x7fe2fc06f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:25.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2f9ffb640 1 -- 172.21.15.40:0/194639657 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2fc06b6b0 con 0x7fe2fc0cce70 2023-10-16T08:39:25.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2f9ffb640 1 -- 172.21.15.40:0/194639657 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7fe2f0001680 con 0x7fe2fc0cce70 2023-10-16T08:39:25.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2f9ffb640 1 -- 172.21.15.40:0/194639657 >> v1:172.21.15.40:6789/0 conn(0x7fe2fc0cce70 legacy=0x7fe2fc0ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:25.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2f9ffb640 1 --2- 172.21.15.40:0/194639657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2e40039f0 0x7fe2e4003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:25.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2f9ffb640 1 --2- 172.21.15.40:0/194639657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2e40047d0 0x7fe2e4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:25.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2f9ffb640 1 --2- 172.21.15.40:0/194639657 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2e4005190 0x7fe2e400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:25.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2f9ffb640 1 -- 172.21.15.40:0/194639657 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc06b3d0 con 0x7fe2e40039f0 2023-10-16T08:39:25.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2f9ffb640 1 -- 172.21.15.40:0/194639657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e400c0c0 con 0x7fe2e40047d0 2023-10-16T08:39:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2f9ffb640 1 -- 172.21.15.40:0/194639657 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e400c280 con 0x7fe2e4005190 2023-10-16T08:39:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2faffd640 1 --2- 172.21.15.40:0/194639657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2e40039f0 0x7fe2e4003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:25.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2fb7fe640 1 --2- 172.21.15.40:0/194639657 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2e4005190 0x7fe2e400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2fa7fc640 1 --2- 172.21.15.40:0/194639657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2e40047d0 0x7fe2e4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2faffd640 1 -- 172.21.15.40:0/194639657 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2e4005190 msgr2=0x7fe2e400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2faffd640 1 --2- 172.21.15.40:0/194639657 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2e4005190 0x7fe2e400b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2faffd640 1 -- 172.21.15.40:0/194639657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2e40047d0 msgr2=0x7fe2e4004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2faffd640 1 --2- 172.21.15.40:0/194639657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2e40047d0 0x7fe2e4004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2faffd640 1 -- 172.21.15.40:0/194639657 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e4003530 con 0x7fe2e40039f0 2023-10-16T08:39:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2fb7fe640 1 -- 172.21.15.40:0/194639657 reap_dead start 2023-10-16T08:39:25.080 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.063+0000 7fe2fa7fc640 1 --2- 172.21.15.40:0/194639657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2e40047d0 0x7fe2e4004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.067+0000 7fe2faffd640 1 --2- 172.21.15.40:0/194639657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2e40039f0 0x7fe2e4003e10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec001780 tx=0x7fe2ec01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c441c04ce06c1569 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.067+0000 7fe2f9ffb640 1 -- 172.21.15.40:0/194639657 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec0052d0 con 0x7fe2e40039f0 2023-10-16T08:39:25.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.067+0000 7fe2f9ffb640 1 -- 172.21.15.40:0/194639657 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2ec0a7050 con 0x7fe2e40039f0 2023-10-16T08:39:25.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.067+0000 7fe2f9ffb640 1 -- 172.21.15.40:0/194639657 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ec003600 con 0x7fe2e40039f0 2023-10-16T08:39:25.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.067+0000 7fe3012b3640 1 -- 172.21.15.40:0/194639657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2e40039f0 msgr2=0x7fe2e4003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:25.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.067+0000 7fe3012b3640 1 --2- 172.21.15.40:0/194639657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2e40039f0 0x7fe2e4003e10 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec001780 tx=0x7fe2ec01c3d0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.067+0000 7fe3012b3640 1 -- 172.21.15.40:0/194639657 shutdown_connections 2023-10-16T08:39:25.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.067+0000 7fe3012b3640 1 --2- 172.21.15.40:0/194639657 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2e40039f0 0x7fe2e4003e10 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.067+0000 7fe3012b3640 1 -- 172.21.15.40:0/194639657 >> 172.21.15.40:0/194639657 conn(0x7fe2fc0ce890 msgr2=0x7fe2fc0ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:25.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.067+0000 7fe3012b3640 1 -- 172.21.15.40:0/194639657 shutdown_connections 2023-10-16T08:39:25.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.067+0000 7fe3012b3640 1 -- 172.21.15.40:0/194639657 wait complete. 2023-10-16T08:39:25.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.067+0000 7fe3012b3640 1 Processor -- start 2023-10-16T08:39:25.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.067+0000 7fe3012b3640 1 -- start start 2023-10-16T08:39:25.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe3012b3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2e40039f0 0x7fe2fc156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:25.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe3012b3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2fc15e8b0 0x7fe2fc0b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:25.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe3012b3640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2fc15ed30 0x7fe2fc0b0e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:25.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe3012b3640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc15d810 con 0x7fe2fc15e8b0 2023-10-16T08:39:25.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe3012b3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc15daf0 con 0x7fe2e40039f0 2023-10-16T08:39:25.089 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe3012b3640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2fc15ddd0 con 0x7fe2fc15ed30 2023-10-16T08:39:25.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe2fa7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2fc15e8b0 0x7fe2fc0b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:25.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe2fa7fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2fc15e8b0 0x7fe2fc0b08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49748/0 (socket says 172.21.15.40:49748) 2023-10-16T08:39:25.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe2fa7fc640 1 -- 172.21.15.40:0/2493102343 learned_addr learned my addr 172.21.15.40:0/2493102343 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:25.090 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe2faffd640 1 --2- 172.21.15.40:0/2493102343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2e40039f0 0x7fe2fc156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:25.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe2fb7fe640 1 --2- 172.21.15.40:0/2493102343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2fc15ed30 0x7fe2fc0b0e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:25.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe2fa7fc640 1 -- 172.21.15.40:0/2493102343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2fc15ed30 msgr2=0x7fe2fc0b0e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:25.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe2fa7fc640 1 --2- 172.21.15.40:0/2493102343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2fc15ed30 0x7fe2fc0b0e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.091 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe2fa7fc640 1 -- 172.21.15.40:0/2493102343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2e40039f0 msgr2=0x7fe2fc156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:25.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe2fa7fc640 1 --2- 172.21.15.40:0/2493102343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2e40039f0 0x7fe2fc156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe2fa7fc640 1 -- 172.21.15.40:0/2493102343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2fc15e0b0 con 0x7fe2fc15e8b0 2023-10-16T08:39:25.092 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe2fa7fc640 1 --2- 172.21.15.40:0/2493102343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2fc15e8b0 0x7fe2fc0b08c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00018f0 tx=0x7fe2f000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:25.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe2e37fe640 1 -- 172.21.15.40:0/2493102343 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f00072d0 con 0x7fe2fc15e8b0 2023-10-16T08:39:25.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe2e37fe640 1 -- 172.21.15.40:0/2493102343 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2f0007430 con 0x7fe2fc15e8b0 2023-10-16T08:39:25.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe2e37fe640 1 -- 172.21.15.40:0/2493102343 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2f00047f0 con 0x7fe2fc15e8b0 2023-10-16T08:39:25.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe3012b3640 1 -- 172.21.15.40:0/2493102343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2fc15e390 con 0x7fe2fc15e8b0 2023-10-16T08:39:25.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe3012b3640 1 -- 172.21.15.40:0/2493102343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2fc15faa0 con 0x7fe2fc15e8b0 2023-10-16T08:39:25.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe3012b3640 1 -- 172.21.15.40:0/2493102343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2fc067a20 con 0x7fe2fc15e8b0 2023-10-16T08:39:25.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe2e37fe640 1 -- 172.21.15.40:0/2493102343 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe2f0004950 con 0x7fe2fc15e8b0 2023-10-16T08:39:25.094 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe2e37fe640 1 --2- 172.21.15.40:0/2493102343 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe2e80664d0 0x7fe2e8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:25.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe2e37fe640 1 -- 172.21.15.40:0/2493102343 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2f0083520 con 0x7fe2fc15e8b0 2023-10-16T08:39:25.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.071+0000 7fe2faffd640 1 --2- 172.21.15.40:0/2493102343 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe2e80664d0 0x7fe2e8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:25.095 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.075+0000 7fe2faffd640 1 --2- 172.21.15.40:0/2493102343 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe2e80664d0 0x7fe2e8068990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec001750 tx=0x7fe2ec0011a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:25.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.075+0000 7fe2e37fe640 1 -- 172.21.15.40:0/2493102343 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fe2f0050a30 con 0x7fe2fc15e8b0 2023-10-16T08:39:25.239 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.235+0000 7fe3012b3640 1 -- 172.21.15.40:0/2493102343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe2fc04e9c0 con 0x7fe2fc15e8b0 2023-10-16T08:39:25.240 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.235+0000 7fe2e37fe640 1 -- 172.21.15.40:0/2493102343 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v20) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe2f0054050 con 0x7fe2fc15e8b0 2023-10-16T08:39:25.240 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-16T08:39:25.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.235+0000 7fe3012b3640 1 -- 172.21.15.40:0/2493102343 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe2e80664d0 msgr2=0x7fe2e8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:25.242 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.235+0000 7fe3012b3640 1 --2- 172.21.15.40:0/2493102343 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe2e80664d0 0x7fe2e8068990 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe2ec001750 tx=0x7fe2ec0011a0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.235+0000 7fe3012b3640 1 -- 172.21.15.40:0/2493102343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2fc15e8b0 msgr2=0x7fe2fc0b08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:25.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.235+0000 7fe3012b3640 1 --2- 172.21.15.40:0/2493102343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2fc15e8b0 0x7fe2fc0b08c0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe2f00018f0 tx=0x7fe2f000dc00 comp rx=0 tx=0).stop 2023-10-16T08:39:25.243 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.235+0000 7fe3012b3640 1 -- 172.21.15.40:0/2493102343 shutdown_connections 2023-10-16T08:39:25.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.235+0000 7fe3012b3640 1 --2- 172.21.15.40:0/2493102343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe2fc15ed30 0x7fe2fc0b0e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.235+0000 7fe3012b3640 1 --2- 172.21.15.40:0/2493102343 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe2e80664d0 0x7fe2e8068990 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.235+0000 7fe3012b3640 1 --2- 172.21.15.40:0/2493102343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe2fc15e8b0 0x7fe2fc0b08c0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.235+0000 7fe3012b3640 1 --2- 172.21.15.40:0/2493102343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe2e40039f0 0x7fe2fc156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.235+0000 7fe3012b3640 1 -- 172.21.15.40:0/2493102343 >> 172.21.15.40:0/2493102343 conn(0x7fe2fc0ce890 msgr2=0x7fe2fc06caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:25.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.235+0000 7fe3012b3640 1 -- 172.21.15.40:0/2493102343 shutdown_connections 2023-10-16T08:39:25.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.235+0000 7fe3012b3640 1 -- 172.21.15.40:0/2493102343 wait complete. 2023-10-16T08:39:25.259 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:39:25.387 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.383+0000 7ff9f2ccc640 1 Processor -- start 2023-10-16T08:39:25.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.383+0000 7ff9f2ccc640 1 -- start start 2023-10-16T08:39:25.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.383+0000 7ff9f2ccc640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff9ec0cae10 0x7ff9ec0b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:25.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.383+0000 7ff9f2ccc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ec0c98b0 con 0x7ff9ec0adad0 2023-10-16T08:39:25.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.383+0000 7ff9f2ccc640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9ec0b47c0 con 0x7ff9ec0adeb0 2023-10-16T08:39:25.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.383+0000 7ff9f2ccc640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9ec0c8f80 con 0x7ff9ec0cae10 2023-10-16T08:39:25.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.383+0000 7ff9f0a68640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7ff9ec0adad0 0x7ff9ec0ca700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:36994/0 (socket says 172.21.15.40:36994) 2023-10-16T08:39:25.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.383+0000 7ff9f0a68640 1 -- 172.21.15.40:0/1344758319 learned_addr learned my addr 172.21.15.40:0/1344758319 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:25.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.383+0000 7ff9ebfff640 1 --2- 172.21.15.40:0/1344758319 >> v2:172.21.15.86:3300/0 conn(0x7ff9ec0cae10 0x7ff9ec0b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:25.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.383+0000 7ff9eb7fe640 1 -- 172.21.15.40:0/1344758319 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1812242744 0 0) 0x7ff9ec0c98b0 con 0x7ff9ec0adad0 2023-10-16T08:39:25.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.383+0000 7ff9eb7fe640 1 -- 172.21.15.40:0/1344758319 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff9d4003530 con 0x7ff9ec0adad0 2023-10-16T08:39:25.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.383+0000 7ff9ebfff640 1 -- 172.21.15.40:0/1344758319 >> v1:172.21.15.86:6789/0 conn(0x7ff9ec0adeb0 legacy=0x7ff9ec15a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:25.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.383+0000 7ff9eb7fe640 1 -- 172.21.15.40:0/1344758319 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1878036236 0 0) 0x7ff9d4003530 con 0x7ff9ec0adad0 2023-10-16T08:39:25.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.383+0000 7ff9ebfff640 1 -- 172.21.15.40:0/1344758319 >> v1:172.21.15.40:6789/0 conn(0x7ff9ec0adad0 legacy=0x7ff9ec0ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:25.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.383+0000 7ff9ebfff640 1 -- 172.21.15.40:0/1344758319 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9ec0c9260 con 0x7ff9ec0cae10 2023-10-16T08:39:25.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.387+0000 7ff9ebfff640 1 --2- 172.21.15.40:0/1344758319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9ec0cae10 0x7ff9ec0b41e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00097c0 tx=0x7ff9e001c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7973ac5dcca5b1ff server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:25.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.387+0000 7ff9eb7fe640 1 -- 172.21.15.40:0/1344758319 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9dc001a00 con 0x7ff9ec0cae10 2023-10-16T08:39:25.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.387+0000 7ff9eb7fe640 1 -- 172.21.15.40:0/1344758319 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e00ac440 con 0x7ff9ec0cae10 2023-10-16T08:39:25.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.387+0000 7ff9eb7fe640 1 -- 172.21.15.40:0/1344758319 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e00ac7b0 con 0x7ff9ec0cae10 2023-10-16T08:39:25.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.387+0000 7ff9f2ccc640 1 -- 172.21.15.40:0/1344758319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9ec0cae10 msgr2=0x7ff9ec0b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:25.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.387+0000 7ff9f2ccc640 1 --2- 172.21.15.40:0/1344758319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9ec0cae10 0x7ff9ec0b41e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00097c0 tx=0x7ff9e001c370 comp rx=0 tx=0).stop 2023-10-16T08:39:25.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.387+0000 7ff9f2ccc640 1 -- 172.21.15.40:0/1344758319 shutdown_connections 2023-10-16T08:39:25.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.387+0000 7ff9f2ccc640 1 --2- 172.21.15.40:0/1344758319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9ec0cae10 0x7ff9ec0b41e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.387+0000 7ff9f2ccc640 1 -- 172.21.15.40:0/1344758319 >> 172.21.15.40:0/1344758319 conn(0x7ff9ec0ce890 msgr2=0x7ff9ec0ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:25.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.387+0000 7ff9f2ccc640 1 -- 172.21.15.40:0/1344758319 shutdown_connections 2023-10-16T08:39:25.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.387+0000 7ff9f2ccc640 1 -- 172.21.15.40:0/1344758319 wait complete. 2023-10-16T08:39:25.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.387+0000 7ff9f2ccc640 1 Processor -- start 2023-10-16T08:39:25.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.387+0000 7ff9f2ccc640 1 -- start start 2023-10-16T08:39:25.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9f2ccc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9ec0adad0 0x7ff9ec14dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:25.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9f2ccc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9ec0adeb0 0x7ff9ec14efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:25.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9f2ccc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9ec0cae10 0x7ff9ec14c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:25.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9f2ccc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec0ccec0 con 0x7ff9ec0adeb0 2023-10-16T08:39:25.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9f2ccc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec14fdf0 con 0x7ff9ec0adad0 2023-10-16T08:39:25.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9f2ccc640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9ec14f690 con 0x7ff9ec0cae10 2023-10-16T08:39:25.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9ebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9ec0adeb0 0x7ff9ec14efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:25.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9ebfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9ec0adeb0 0x7ff9ec14efd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49754/0 (socket says 172.21.15.40:49754) 2023-10-16T08:39:25.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9ebfff640 1 -- 172.21.15.40:0/3416190639 learned_addr learned my addr 172.21.15.40:0/3416190639 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:25.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9f0a68640 1 --2- 172.21.15.40:0/3416190639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9ec0adad0 0x7ff9ec14dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:25.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9f1269640 1 --2- 172.21.15.40:0/3416190639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9ec0cae10 0x7ff9ec14c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:25.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9ebfff640 1 -- 172.21.15.40:0/3416190639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9ec0cae10 msgr2=0x7ff9ec14c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:25.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9ebfff640 1 --2- 172.21.15.40:0/3416190639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9ec0cae10 0x7ff9ec14c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9ebfff640 1 -- 172.21.15.40:0/3416190639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9ec0adad0 msgr2=0x7ff9ec14dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:25.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9ebfff640 1 --2- 172.21.15.40:0/3416190639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9ec0adad0 0x7ff9ec14dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9ebfff640 1 -- 172.21.15.40:0/3416190639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e0009420 con 0x7ff9ec0adeb0 2023-10-16T08:39:25.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9ebfff640 1 --2- 172.21.15.40:0/3416190639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9ec0adeb0 0x7ff9ec14efd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00af040 tx=0x7ff9e0019110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:25.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9e97fa640 1 -- 172.21.15.40:0/3416190639 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e0003af0 con 0x7ff9ec0adeb0 2023-10-16T08:39:25.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9e97fa640 1 -- 172.21.15.40:0/3416190639 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9e0003c50 con 0x7ff9ec0adeb0 2023-10-16T08:39:25.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9f2ccc640 1 -- 172.21.15.40:0/3416190639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9ec0cc590 con 0x7ff9ec0adeb0 2023-10-16T08:39:25.406 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9e97fa640 1 -- 172.21.15.40:0/3416190639 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9e00ac5b0 con 0x7ff9ec0adeb0 2023-10-16T08:39:25.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9f2ccc640 1 -- 172.21.15.40:0/3416190639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9ec0ccb50 con 0x7ff9ec0adeb0 2023-10-16T08:39:25.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9f2ccc640 1 -- 172.21.15.40:0/3416190639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9ec067a20 con 0x7ff9ec0adeb0 2023-10-16T08:39:25.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9e97fa640 1 -- 172.21.15.40:0/3416190639 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff9e00b3040 con 0x7ff9ec0adeb0 2023-10-16T08:39:25.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9e97fa640 1 --2- 172.21.15.40:0/3416190639 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff9d80664d0 0x7ff9d8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:25.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9e97fa640 1 -- 172.21.15.40:0/3416190639 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9e011df00 con 0x7ff9ec0adeb0 2023-10-16T08:39:25.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.391+0000 7ff9f0a68640 1 --2- 172.21.15.40:0/3416190639 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff9d80664d0 0x7ff9d8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:25.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.395+0000 7ff9f0a68640 1 --2- 172.21.15.40:0/3416190639 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff9d80664d0 0x7ff9d8068990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0012f0 tx=0x7ff9dc009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:25.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.395+0000 7ff9e97fa640 1 -- 172.21.15.40:0/3416190639 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7ff9e00eb370 con 0x7ff9ec0adeb0 2023-10-16T08:39:25.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.551+0000 7ff9f2ccc640 1 -- 172.21.15.40:0/3416190639 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff9ec14c640 con 0x7ff9ec0adeb0 2023-10-16T08:39:25.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.555+0000 7ff9e97fa640 1 -- 172.21.15.40:0/3416190639 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v20) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff9e00b10a0 con 0x7ff9ec0adeb0 2023-10-16T08:39:25.560 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-16T08:39:25.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.555+0000 7ff9f2ccc640 1 -- 172.21.15.40:0/3416190639 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff9d80664d0 msgr2=0x7ff9d8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:25.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.555+0000 7ff9f2ccc640 1 --2- 172.21.15.40:0/3416190639 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff9d80664d0 0x7ff9d8068990 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff9dc0012f0 tx=0x7ff9dc009040 comp rx=0 tx=0).stop 2023-10-16T08:39:25.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.555+0000 7ff9f2ccc640 1 -- 172.21.15.40:0/3416190639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9ec0adeb0 msgr2=0x7ff9ec14efd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:25.562 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.555+0000 7ff9f2ccc640 1 --2- 172.21.15.40:0/3416190639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9ec0adeb0 0x7ff9ec14efd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7ff9e00af040 tx=0x7ff9e0019110 comp rx=0 tx=0).stop 2023-10-16T08:39:25.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.555+0000 7ff9f2ccc640 1 -- 172.21.15.40:0/3416190639 shutdown_connections 2023-10-16T08:39:25.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.555+0000 7ff9f2ccc640 1 --2- 172.21.15.40:0/3416190639 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff9ec0cae10 0x7ff9ec14c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.563 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.555+0000 7ff9f2ccc640 1 --2- 172.21.15.40:0/3416190639 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff9d80664d0 0x7ff9d8068990 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.555+0000 7ff9f2ccc640 1 --2- 172.21.15.40:0/3416190639 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff9ec0adeb0 0x7ff9ec14efd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.555+0000 7ff9f2ccc640 1 --2- 172.21.15.40:0/3416190639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9ec0adad0 0x7ff9ec14dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.555+0000 7ff9f2ccc640 1 -- 172.21.15.40:0/3416190639 >> 172.21.15.40:0/3416190639 conn(0x7ff9ec0ce890 msgr2=0x7ff9ec1568f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:25.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.555+0000 7ff9f2ccc640 1 -- 172.21.15.40:0/3416190639 shutdown_connections 2023-10-16T08:39:25.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.555+0000 7ff9f2ccc640 1 -- 172.21.15.40:0/3416190639 wait complete. 2023-10-16T08:39:25.579 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:39:25.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.699+0000 7f9ba9483640 1 Processor -- start 2023-10-16T08:39:25.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.699+0000 7f9ba9483640 1 -- start start 2023-10-16T08:39:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.699+0000 7f9ba9483640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9ba40cd960 0x7f9ba40cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.699+0000 7f9ba9483640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ba40d0d40 0x7f9ba40d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.699+0000 7f9ba9483640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ba40ce300 con 0x7f9ba415cce0 2023-10-16T08:39:25.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.699+0000 7f9ba9483640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ba40b7f80 con 0x7f9ba40cd960 2023-10-16T08:39:25.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.699+0000 7f9ba9483640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba40b7370 con 0x7f9ba40d0d40 2023-10-16T08:39:25.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.699+0000 7f9ba37fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ba40d0d40 0x7f9ba40d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:25.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.699+0000 7f9ba27fc640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9ba40cd960 0x7f9ba40cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:25.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.699+0000 7f9ba2ffd640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f9ba415cce0 0x7f9ba40b68d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.40:54890/0 (socket says 172.21.15.40:54890) 2023-10-16T08:39:25.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.699+0000 7f9ba27fc640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f9ba40cd960 0x7f9ba40cddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.40:45892/0 (socket says 172.21.15.40:45892) 2023-10-16T08:39:25.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.699+0000 7f9ba2ffd640 1 -- 172.21.15.40:0/777639829 learned_addr learned my addr 172.21.15.40:0/777639829 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.699+0000 7f9ba27fc640 1 -- 172.21.15.40:0/777639829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ba40d0d40 msgr2=0x7f9ba40d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.699+0000 7f9ba27fc640 1 --2- 172.21.15.40:0/777639829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ba40d0d40 0x7f9ba40d11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.699+0000 7f9ba27fc640 1 -- 172.21.15.40:0/777639829 >> v1:172.21.15.86:6789/0 conn(0x7f9ba415cce0 legacy=0x7f9ba40b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.699+0000 7f9ba27fc640 1 -- 172.21.15.40:0/777639829 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ba40b7650 con 0x7f9ba40cd960 2023-10-16T08:39:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.703+0000 7f9ba27fc640 1 --2- 172.21.15.40:0/777639829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ba40cd960 0x7f9ba40cddc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9b980097c0 tx=0x7f9b9801c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=544480e269c28602 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:25.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.703+0000 7f9ba1ffb640 1 -- 172.21.15.40:0/777639829 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b980ab600 con 0x7f9ba40cd960 2023-10-16T08:39:25.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.703+0000 7f9ba1ffb640 1 -- 172.21.15.40:0/777639829 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b980a7050 con 0x7f9ba40cd960 2023-10-16T08:39:25.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.703+0000 7f9ba1ffb640 1 -- 172.21.15.40:0/777639829 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b980ab980 con 0x7f9ba40cd960 2023-10-16T08:39:25.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.703+0000 7f9ba9483640 1 -- 172.21.15.40:0/777639829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ba40cd960 msgr2=0x7f9ba40cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:25.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.703+0000 7f9ba9483640 1 --2- 172.21.15.40:0/777639829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ba40cd960 0x7f9ba40cddc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f9b980097c0 tx=0x7f9b9801c310 comp rx=0 tx=0).stop 2023-10-16T08:39:25.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.703+0000 7f9ba9483640 1 -- 172.21.15.40:0/777639829 shutdown_connections 2023-10-16T08:39:25.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.703+0000 7f9ba9483640 1 --2- 172.21.15.40:0/777639829 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ba40d0d40 0x7f9ba40d11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.712 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.703+0000 7f9ba9483640 1 --2- 172.21.15.40:0/777639829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ba40cd960 0x7f9ba40cddc0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.703+0000 7f9ba9483640 1 -- 172.21.15.40:0/777639829 >> 172.21.15.40:0/777639829 conn(0x7f9ba406a290 msgr2=0x7f9ba406a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.703+0000 7f9ba9483640 1 -- 172.21.15.40:0/777639829 shutdown_connections 2023-10-16T08:39:25.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.703+0000 7f9ba9483640 1 -- 172.21.15.40:0/777639829 wait complete. 2023-10-16T08:39:25.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.703+0000 7f9ba9483640 1 Processor -- start 2023-10-16T08:39:25.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.703+0000 7f9ba9483640 1 -- start start 2023-10-16T08:39:25.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.703+0000 7f9ba9483640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ba40cd960 0x7f9ba4150440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:25.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.703+0000 7f9ba9483640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ba40d0d40 0x7f9ba414ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:25.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9ba9483640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ba415cce0 0x7f9ba415d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:25.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9ba9483640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba4150020 con 0x7f9ba415cce0 2023-10-16T08:39:25.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9ba9483640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba414f3a0 con 0x7f9ba40cd960 2023-10-16T08:39:25.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9ba9483640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ba4160450 con 0x7f9ba40d0d40 2023-10-16T08:39:25.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9ba37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ba415cce0 0x7f9ba415d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:25.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9ba37fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ba415cce0 0x7f9ba415d390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49764/0 (socket says 172.21.15.40:49764) 2023-10-16T08:39:25.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9ba37fe640 1 -- 172.21.15.40:0/3742607967 learned_addr learned my addr 172.21.15.40:0/3742607967 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:25.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9ba27fc640 1 --2- 172.21.15.40:0/3742607967 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ba40d0d40 0x7f9ba414ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:25.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9ba2ffd640 1 --2- 172.21.15.40:0/3742607967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ba40cd960 0x7f9ba4150440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:25.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9ba37fe640 1 -- 172.21.15.40:0/3742607967 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ba40d0d40 msgr2=0x7f9ba414ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:25.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9ba37fe640 1 --2- 172.21.15.40:0/3742607967 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ba40d0d40 0x7f9ba414ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9ba37fe640 1 -- 172.21.15.40:0/3742607967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ba40cd960 msgr2=0x7f9ba4150440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:25.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9ba37fe640 1 --2- 172.21.15.40:0/3742607967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ba40cd960 0x7f9ba4150440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9ba37fe640 1 -- 172.21.15.40:0/3742607967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b98009420 con 0x7f9ba415cce0 2023-10-16T08:39:25.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9ba37fe640 1 --2- 172.21.15.40:0/3742607967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ba415cce0 0x7f9ba415d390 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00efc0 tx=0x7f9b9c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:25.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9b8b7fe640 1 -- 172.21.15.40:0/3742607967 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c014480 con 0x7f9ba415cce0 2023-10-16T08:39:25.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9b8b7fe640 1 -- 172.21.15.40:0/3742607967 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9b9c00f040 con 0x7f9ba415cce0 2023-10-16T08:39:25.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9b8b7fe640 1 -- 172.21.15.40:0/3742607967 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9b9c014820 con 0x7f9ba415cce0 2023-10-16T08:39:25.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9ba9483640 1 -- 172.21.15.40:0/3742607967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ba414f680 con 0x7f9ba415cce0 2023-10-16T08:39:25.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9ba9483640 1 -- 172.21.15.40:0/3742607967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ba414fc40 con 0x7f9ba415cce0 2023-10-16T08:39:25.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9ba9483640 1 -- 172.21.15.40:0/3742607967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ba4150160 con 0x7f9ba415cce0 2023-10-16T08:39:25.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9b8b7fe640 1 -- 172.21.15.40:0/3742607967 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9b9c014980 con 0x7f9ba415cce0 2023-10-16T08:39:25.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9b8b7fe640 1 --2- 172.21.15.40:0/3742607967 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f9b900664a0 0x7f9b90068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:25.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9b8b7fe640 1 -- 172.21.15.40:0/3742607967 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9b9c084da0 con 0x7f9ba415cce0 2023-10-16T08:39:25.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.707+0000 7f9ba2ffd640 1 --2- 172.21.15.40:0/3742607967 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f9b900664a0 0x7f9b90068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:25.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.711+0000 7f9ba2ffd640 1 --2- 172.21.15.40:0/3742607967 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f9b900664a0 0x7f9b90068960 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9b94000f70 tx=0x7f9b94007e30 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:25.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.711+0000 7f9b8b7fe640 1 -- 172.21.15.40:0/3742607967 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f9b9c052440 con 0x7f9ba415cce0 2023-10-16T08:39:25.891 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.887+0000 7f9ba9483640 1 -- 172.21.15.40:0/3742607967 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9ba4150160 con 0x7f9ba415cce0 2023-10-16T08:39:25.892 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.887+0000 7f9b8b7fe640 1 -- 172.21.15.40:0/3742607967 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v20) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9ba4150160 con 0x7f9ba415cce0 2023-10-16T08:39:25.893 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-16T08:39:25.894 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.887+0000 7f9ba9483640 1 -- 172.21.15.40:0/3742607967 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f9b900664a0 msgr2=0x7f9b90068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:25.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.887+0000 7f9ba9483640 1 --2- 172.21.15.40:0/3742607967 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f9b900664a0 0x7f9b90068960 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f9b94000f70 tx=0x7f9b94007e30 comp rx=0 tx=0).stop 2023-10-16T08:39:25.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.887+0000 7f9ba9483640 1 -- 172.21.15.40:0/3742607967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ba415cce0 msgr2=0x7f9ba415d390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:25.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.887+0000 7f9ba9483640 1 --2- 172.21.15.40:0/3742607967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ba415cce0 0x7f9ba415d390 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f9b9c00efc0 tx=0x7f9b9c00c490 comp rx=0 tx=0).stop 2023-10-16T08:39:25.895 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.887+0000 7f9ba9483640 1 -- 172.21.15.40:0/3742607967 shutdown_connections 2023-10-16T08:39:25.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.887+0000 7f9ba9483640 1 --2- 172.21.15.40:0/3742607967 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f9b900664a0 0x7f9b90068960 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.887+0000 7f9ba9483640 1 --2- 172.21.15.40:0/3742607967 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f9ba415cce0 0x7f9ba415d390 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.896 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.887+0000 7f9ba9483640 1 --2- 172.21.15.40:0/3742607967 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f9ba40d0d40 0x7f9ba414ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.887+0000 7f9ba9483640 1 --2- 172.21.15.40:0/3742607967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ba40cd960 0x7f9ba4150440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:25.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.887+0000 7f9ba9483640 1 -- 172.21.15.40:0/3742607967 >> 172.21.15.40:0/3742607967 conn(0x7f9ba406a290 msgr2=0x7f9ba4156740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:25.897 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.887+0000 7f9ba9483640 1 -- 172.21.15.40:0/3742607967 shutdown_connections 2023-10-16T08:39:25.898 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:25.887+0000 7f9ba9483640 1 -- 172.21.15.40:0/3742607967 wait complete. 2023-10-16T08:39:25.911 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2023-10-16T08:39:25.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:25.912 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-16T08:39:25.938 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi086.stderr:ceph-fuse[44060]: fuse finished with error 0 and tester_r 0 2023-10-16T08:39:25.940 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:39:32.042 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-16T08:39:32.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:32.043 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-16T08:39:32.068 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:39:32.068 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:39:32.115 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:39:32.115 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:39:32.163 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:39:32.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d9a2c640 1 Processor -- start 2023-10-16T08:39:32.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d9a2c640 1 -- start start 2023-10-16T08:39:32.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d9a2c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38d40adec0 0x7f38d406bec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d9a2c640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d40adca0 con 0x7f38d40cae50 2023-10-16T08:39:32.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d9a2c640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38d40c5840 con 0x7f38d415b040 2023-10-16T08:39:32.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d9a2c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d40cf890 con 0x7f38d40adec0 2023-10-16T08:39:32.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d27fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f38d40cae50 0x7f38d406b7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37006/0 (socket says 172.21.15.40:37006) 2023-10-16T08:39:32.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d27fc640 1 -- 172.21.15.40:0/2410700259 learned_addr learned my addr 172.21.15.40:0/2410700259 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:32.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d1ffb640 1 -- 172.21.15.40:0/2410700259 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2967751390 0 0) 0x7f38d40adca0 con 0x7f38d40cae50 2023-10-16T08:39:32.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d37fe640 1 --2- 172.21.15.40:0/2410700259 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38d40adec0 0x7f38d406bec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d1ffb640 1 -- 172.21.15.40:0/2410700259 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38bc003530 con 0x7f38d40cae50 2023-10-16T08:39:32.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d1ffb640 1 -- 172.21.15.40:0/2410700259 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4014077587 0 0) 0x7f38bc003530 con 0x7f38d40cae50 2023-10-16T08:39:32.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d1ffb640 1 -- 172.21.15.40:0/2410700259 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38d40adec0 msgr2=0x7f38d406bec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:32.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d1ffb640 1 --2- 172.21.15.40:0/2410700259 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38d40adec0 0x7f38d406bec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d1ffb640 1 -- 172.21.15.40:0/2410700259 >> v1:172.21.15.86:6789/0 conn(0x7f38d415b040 legacy=0x7f38d40ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:32.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d1ffb640 1 -- 172.21.15.40:0/2410700259 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d40cf5b0 con 0x7f38d40cae50 2023-10-16T08:39:32.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d1ffb640 1 -- 172.21.15.40:0/2410700259 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f38c4001680 con 0x7f38d40cae50 2023-10-16T08:39:32.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d1ffb640 1 -- 172.21.15.40:0/2410700259 >> v1:172.21.15.40:6789/0 conn(0x7f38d40cae50 legacy=0x7f38d406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:32.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d1ffb640 1 --2- 172.21.15.40:0/2410700259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38bc003a20 0x7f38bc003e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d1ffb640 1 --2- 172.21.15.40:0/2410700259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc0047d0 0x7f38bc004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d1ffb640 1 --2- 172.21.15.40:0/2410700259 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38d40adec0 0x7f38bc00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d1ffb640 1 -- 172.21.15.40:0/2410700259 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d40cf890 con 0x7f38bc0047d0 2023-10-16T08:39:32.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d1ffb640 1 -- 172.21.15.40:0/2410700259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc0055d0 con 0x7f38bc003a20 2023-10-16T08:39:32.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d1ffb640 1 -- 172.21.15.40:0/2410700259 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38bc00bed0 con 0x7f38d40adec0 2023-10-16T08:39:32.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d27fc640 1 --2- 172.21.15.40:0/2410700259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc0047d0 0x7f38bc004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.304 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d2ffd640 1 --2- 172.21.15.40:0/2410700259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38bc003a20 0x7f38bc003e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d27fc640 1 -- 172.21.15.40:0/2410700259 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38d40adec0 msgr2=0x7f38bc00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:32.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d27fc640 1 --2- 172.21.15.40:0/2410700259 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38d40adec0 0x7f38bc00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d27fc640 1 -- 172.21.15.40:0/2410700259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38bc003a20 msgr2=0x7f38bc003e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:32.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d27fc640 1 --2- 172.21.15.40:0/2410700259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38bc003a20 0x7f38bc003e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.291+0000 7f38d27fc640 1 -- 172.21.15.40:0/2410700259 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38bc003530 con 0x7f38bc0047d0 2023-10-16T08:39:32.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.295+0000 7f38d27fc640 1 --2- 172.21.15.40:0/2410700259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc0047d0 0x7f38bc004c50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f38c400c240 tx=0x7f38c401cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a46c0da8523ba068 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:32.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.295+0000 7f38d1ffb640 1 -- 172.21.15.40:0/2410700259 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c4005ae0 con 0x7f38bc0047d0 2023-10-16T08:39:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.295+0000 7f38d1ffb640 1 -- 172.21.15.40:0/2410700259 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c4005c40 con 0x7f38bc0047d0 2023-10-16T08:39:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.295+0000 7f38d1ffb640 1 -- 172.21.15.40:0/2410700259 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c40a5800 con 0x7f38bc0047d0 2023-10-16T08:39:32.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.295+0000 7f38d9a2c640 1 -- 172.21.15.40:0/2410700259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc0047d0 msgr2=0x7f38bc004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:32.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.295+0000 7f38d9a2c640 1 --2- 172.21.15.40:0/2410700259 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38bc0047d0 0x7f38bc004c50 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f38c400c240 tx=0x7f38c401cc10 comp rx=0 tx=0).stop 2023-10-16T08:39:32.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.295+0000 7f38d37fe640 1 -- 172.21.15.40:0/2410700259 reap_dead start 2023-10-16T08:39:32.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.295+0000 7f38d9a2c640 1 -- 172.21.15.40:0/2410700259 shutdown_connections 2023-10-16T08:39:32.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.295+0000 7f38d9a2c640 1 -- 172.21.15.40:0/2410700259 >> 172.21.15.40:0/2410700259 conn(0x7f38d40ce960 msgr2=0x7f38d40ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:32.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.295+0000 7f38d9a2c640 1 -- 172.21.15.40:0/2410700259 shutdown_connections 2023-10-16T08:39:32.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.295+0000 7f38d9a2c640 1 -- 172.21.15.40:0/2410700259 wait complete. 2023-10-16T08:39:32.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.295+0000 7f38d9a2c640 1 Processor -- start 2023-10-16T08:39:32.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.295+0000 7f38d9a2c640 1 -- start start 2023-10-16T08:39:32.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d9a2c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38d41517a0 0x7f38d40ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d9a2c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38d4151f50 0x7f38d40ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d9a2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38d415db30 0x7f38d415dfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d9a2c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d415e4f0 con 0x7f38d4151f50 2023-10-16T08:39:32.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d9a2c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38d4157730 con 0x7f38d415db30 2023-10-16T08:39:32.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d9a2c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38d40cc5d0 con 0x7f38d41517a0 2023-10-16T08:39:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d27fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38d4151f50 0x7f38d40ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d27fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38d4151f50 0x7f38d40ace20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49776/0 (socket says 172.21.15.40:49776) 2023-10-16T08:39:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d27fc640 1 -- 172.21.15.40:0/3019650654 learned_addr learned my addr 172.21.15.40:0/3019650654 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:32.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d37fe640 1 --2- 172.21.15.40:0/3019650654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38d415db30 0x7f38d415dfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d2ffd640 1 --2- 172.21.15.40:0/3019650654 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38d41517a0 0x7f38d40ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d27fc640 1 -- 172.21.15.40:0/3019650654 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38d41517a0 msgr2=0x7f38d40ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d27fc640 1 --2- 172.21.15.40:0/3019650654 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38d41517a0 0x7f38d40ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d27fc640 1 -- 172.21.15.40:0/3019650654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38d415db30 msgr2=0x7f38d415dfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:32.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d27fc640 1 --2- 172.21.15.40:0/3019650654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38d415db30 0x7f38d415dfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d27fc640 1 -- 172.21.15.40:0/3019650654 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d40cc8b0 con 0x7f38d4151f50 2023-10-16T08:39:32.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d27fc640 1 --2- 172.21.15.40:0/3019650654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38d4151f50 0x7f38d40ace20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f38c4001940 tx=0x7f38bc005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:32.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38bbfff640 1 -- 172.21.15.40:0/3019650654 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c40b4070 con 0x7f38d4151f50 2023-10-16T08:39:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38bbfff640 1 -- 172.21.15.40:0/3019650654 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38c40a5e80 con 0x7f38d4151f50 2023-10-16T08:39:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d9a2c640 1 -- 172.21.15.40:0/3019650654 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38d40ccb90 con 0x7f38d4151f50 2023-10-16T08:39:32.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38bbfff640 1 -- 172.21.15.40:0/3019650654 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38c4001440 con 0x7f38d4151f50 2023-10-16T08:39:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d9a2c640 1 -- 172.21.15.40:0/3019650654 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38d40cd150 con 0x7f38d4151f50 2023-10-16T08:39:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d9a2c640 1 -- 172.21.15.40:0/3019650654 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38d404e9c0 con 0x7f38d4151f50 2023-10-16T08:39:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38bbfff640 1 -- 172.21.15.40:0/3019650654 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f38c40048b0 con 0x7f38d4151f50 2023-10-16T08:39:32.319 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38bbfff640 1 --2- 172.21.15.40:0/3019650654 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f38c00664a0 0x7f38c0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38bbfff640 1 -- 172.21.15.40:0/3019650654 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f38c411d620 con 0x7f38d4151f50 2023-10-16T08:39:32.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.299+0000 7f38d2ffd640 1 --2- 172.21.15.40:0/3019650654 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f38c00664a0 0x7f38c0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.320 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.303+0000 7f38d2ffd640 1 --2- 172.21.15.40:0/3019650654 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f38c00664a0 0x7f38c0068960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f38c8005510 tx=0x7f38c8009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:32.321 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.303+0000 7f38bbfff640 1 -- 172.21.15.40:0/3019650654 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f38c40ea980 con 0x7f38d4151f50 2023-10-16T08:39:32.467 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.463+0000 7f38d9a2c640 1 -- 172.21.15.40:0/3019650654 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f38d4071cf0 con 0x7f38d4151f50 2023-10-16T08:39:32.470 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.463+0000 7f38bbfff640 1 -- 172.21.15.40:0/3019650654 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16741 (secure 0 0 0) 0x7f38c40edfa0 con 0x7f38d4151f50 2023-10-16T08:39:32.470 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:39:32.471 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:39:26.948454+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:39:32.472 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.467+0000 7f38d9a2c640 1 -- 172.21.15.40:0/3019650654 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f38c00664a0 msgr2=0x7f38c0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:32.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.467+0000 7f38d9a2c640 1 --2- 172.21.15.40:0/3019650654 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f38c00664a0 0x7f38c0068960 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f38c8005510 tx=0x7f38c8009040 comp rx=0 tx=0).stop 2023-10-16T08:39:32.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.467+0000 7f38d9a2c640 1 -- 172.21.15.40:0/3019650654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38d4151f50 msgr2=0x7f38d40ace20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:32.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.467+0000 7f38d9a2c640 1 --2- 172.21.15.40:0/3019650654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38d4151f50 0x7f38d40ace20 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f38c4001940 tx=0x7f38bc005800 comp rx=0 tx=0).stop 2023-10-16T08:39:32.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.467+0000 7f38d9a2c640 1 -- 172.21.15.40:0/3019650654 shutdown_connections 2023-10-16T08:39:32.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.467+0000 7f38d9a2c640 1 --2- 172.21.15.40:0/3019650654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38d415db30 0x7f38d415dfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.467+0000 7f38d9a2c640 1 --2- 172.21.15.40:0/3019650654 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f38c00664a0 0x7f38c0068960 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.467+0000 7f38d9a2c640 1 --2- 172.21.15.40:0/3019650654 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f38d4151f50 0x7f38d40ace20 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.467+0000 7f38d9a2c640 1 --2- 172.21.15.40:0/3019650654 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f38d41517a0 0x7f38d40ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.467+0000 7f38d9a2c640 1 -- 172.21.15.40:0/3019650654 >> 172.21.15.40:0/3019650654 conn(0x7f38d40ce960 msgr2=0x7f38d40b3820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:32.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.467+0000 7f38d9a2c640 1 -- 172.21.15.40:0/3019650654 shutdown_connections 2023-10-16T08:39:32.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.467+0000 7f38d9a2c640 1 -- 172.21.15.40:0/3019650654 wait complete. 2023-10-16T08:39:32.489 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:38:33.014668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:38:43.055215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:38:43.971666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:39:32.490 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:39:32.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.599+0000 7f6d233b8640 1 Processor -- start 2023-10-16T08:39:32.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d233b8640 1 -- start start 2023-10-16T08:39:32.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d233b8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d1c0adec0 0x7f6d1c0c97f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d233b8640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f6d1c06b800 0x7f6d1c0c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d233b8640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d1c15b490 con 0x7f6d1c159dd0 2023-10-16T08:39:32.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d233b8640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d1c06cd70 con 0x7f6d1c06b800 2023-10-16T08:39:32.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d233b8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c0c9680 con 0x7f6d1c0adec0 2023-10-16T08:39:32.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d21154640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f6d1c159dd0 0x7f6d1c0c8f70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37018/0 (socket says 172.21.15.40:37018) 2023-10-16T08:39:32.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d21154640 1 -- 172.21.15.40:0/438690627 learned_addr learned my addr 172.21.15.40:0/438690627 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:32.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d21955640 1 --2- 172.21.15.40:0/438690627 >> v2:172.21.15.86:3300/0 conn(0x7f6d1c06b800 0x7f6d1c0c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d20953640 1 --2- 172.21.15.40:0/438690627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d1c0adec0 0x7f6d1c0c97f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d13fff640 1 -- 172.21.15.40:0/438690627 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1803183485 0 0) 0x7f6d1c15b490 con 0x7f6d1c159dd0 2023-10-16T08:39:32.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d13fff640 1 -- 172.21.15.40:0/438690627 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6d08003530 con 0x7f6d1c159dd0 2023-10-16T08:39:32.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d13fff640 1 -- 172.21.15.40:0/438690627 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2414998666 0 0) 0x7f6d08003530 con 0x7f6d1c159dd0 2023-10-16T08:39:32.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d13fff640 1 -- 172.21.15.40:0/438690627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d1c0adec0 msgr2=0x7f6d1c0c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:32.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d13fff640 1 --2- 172.21.15.40:0/438690627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d1c0adec0 0x7f6d1c0c97f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d13fff640 1 -- 172.21.15.40:0/438690627 >> v2:172.21.15.86:3300/0 conn(0x7f6d1c06b800 msgr2=0x7f6d1c0c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:32.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d13fff640 1 --2- 172.21.15.40:0/438690627 >> v2:172.21.15.86:3300/0 conn(0x7f6d1c06b800 0x7f6d1c0c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d13fff640 1 -- 172.21.15.40:0/438690627 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c06d230 con 0x7f6d1c159dd0 2023-10-16T08:39:32.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d13fff640 1 -- 172.21.15.40:0/438690627 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f6d18001680 con 0x7f6d1c159dd0 2023-10-16T08:39:32.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d13fff640 1 -- 172.21.15.40:0/438690627 >> v1:172.21.15.40:6789/0 conn(0x7f6d1c159dd0 legacy=0x7f6d1c0c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:32.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d13fff640 1 --2- 172.21.15.40:0/438690627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d1c0adec0 0x7f6d080039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d13fff640 1 --2- 172.21.15.40:0/438690627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d08003f70 0x7f6d080047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d13fff640 1 --2- 172.21.15.40:0/438690627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d08004d10 0x7f6d08005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d13fff640 1 -- 172.21.15.40:0/438690627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c06cd70 con 0x7f6d08003f70 2023-10-16T08:39:32.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d13fff640 1 -- 172.21.15.40:0/438690627 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0800b990 con 0x7f6d08004d10 2023-10-16T08:39:32.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d13fff640 1 -- 172.21.15.40:0/438690627 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0800bb00 con 0x7f6d1c0adec0 2023-10-16T08:39:32.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d20953640 1 --2- 172.21.15.40:0/438690627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d08003f70 0x7f6d080047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d21154640 1 --2- 172.21.15.40:0/438690627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d1c0adec0 0x7f6d080039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d21955640 1 --2- 172.21.15.40:0/438690627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d08004d10 0x7f6d08005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d20953640 1 -- 172.21.15.40:0/438690627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d1c0adec0 msgr2=0x7f6d080039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:32.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d20953640 1 --2- 172.21.15.40:0/438690627 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d1c0adec0 0x7f6d080039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d20953640 1 -- 172.21.15.40:0/438690627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d08004d10 msgr2=0x7f6d08005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:32.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d20953640 1 --2- 172.21.15.40:0/438690627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d08004d10 0x7f6d08005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.603+0000 7f6d20953640 1 -- 172.21.15.40:0/438690627 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d08003530 con 0x7f6d08003f70 2023-10-16T08:39:32.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d20953640 1 --2- 172.21.15.40:0/438690627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d08003f70 0x7f6d080047d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009ab0 tx=0x7f6d0c01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f54eef9e4d753a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:32.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d13fff640 1 -- 172.21.15.40:0/438690627 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c0a5510 con 0x7f6d08003f70 2023-10-16T08:39:32.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d13fff640 1 -- 172.21.15.40:0/438690627 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d0c0a5670 con 0x7f6d08003f70 2023-10-16T08:39:32.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d13fff640 1 -- 172.21.15.40:0/438690627 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d0c0a5a20 con 0x7f6d08003f70 2023-10-16T08:39:32.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d233b8640 1 -- 172.21.15.40:0/438690627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d08003f70 msgr2=0x7f6d080047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:32.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d233b8640 1 --2- 172.21.15.40:0/438690627 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d08003f70 0x7f6d080047d0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c009ab0 tx=0x7f6d0c01ca30 comp rx=0 tx=0).stop 2023-10-16T08:39:32.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d21955640 1 -- 172.21.15.40:0/438690627 reap_dead start 2023-10-16T08:39:32.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d233b8640 1 -- 172.21.15.40:0/438690627 shutdown_connections 2023-10-16T08:39:32.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d233b8640 1 -- 172.21.15.40:0/438690627 >> 172.21.15.40:0/438690627 conn(0x7f6d1c0ce890 msgr2=0x7f6d1c0ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:32.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d233b8640 1 -- 172.21.15.40:0/438690627 shutdown_connections 2023-10-16T08:39:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d233b8640 1 -- 172.21.15.40:0/438690627 wait complete. 2023-10-16T08:39:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d233b8640 1 Processor -- start 2023-10-16T08:39:32.628 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d233b8640 1 -- start start 2023-10-16T08:39:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d233b8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d08003f70 0x7f6d1c1f3f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d233b8640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d1c150520 0x7f6d1c14ed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d233b8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d1c151110 0x7f6d1c1f11f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d233b8640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c1f4740 con 0x7f6d08003f70 2023-10-16T08:39:32.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d233b8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c1f4a20 con 0x7f6d1c151110 2023-10-16T08:39:32.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d233b8640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d1c1f4d00 con 0x7f6d1c150520 2023-10-16T08:39:32.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d21154640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d08003f70 0x7f6d1c1f3f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d21154640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d08003f70 0x7f6d1c1f3f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49800/0 (socket says 172.21.15.40:49800) 2023-10-16T08:39:32.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d21154640 1 -- 172.21.15.40:0/2152257348 learned_addr learned my addr 172.21.15.40:0/2152257348 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:32.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d20953640 1 --2- 172.21.15.40:0/2152257348 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d1c150520 0x7f6d1c14ed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d21955640 1 --2- 172.21.15.40:0/2152257348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d1c151110 0x7f6d1c1f11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d21154640 1 -- 172.21.15.40:0/2152257348 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d1c150520 msgr2=0x7f6d1c14ed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:32.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d21154640 1 --2- 172.21.15.40:0/2152257348 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d1c150520 0x7f6d1c14ed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d21154640 1 -- 172.21.15.40:0/2152257348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d1c151110 msgr2=0x7f6d1c1f11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:32.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d21154640 1 --2- 172.21.15.40:0/2152257348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d1c151110 0x7f6d1c1f11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.633 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d21154640 1 -- 172.21.15.40:0/2152257348 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d1c1f0cd0 con 0x7f6d08003f70 2023-10-16T08:39:32.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d21955640 1 --2- 172.21.15.40:0/2152257348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d1c151110 0x7f6d1c1f11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:32.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.607+0000 7f6d21154640 1 --2- 172.21.15.40:0/2152257348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d08003f70 0x7f6d1c1f3f10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6d18011840 tx=0x7f6d18011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:32.634 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.611+0000 7f6d11ffb640 1 -- 172.21.15.40:0/2152257348 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1800fb60 con 0x7f6d08003f70 2023-10-16T08:39:32.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.611+0000 7f6d11ffb640 1 -- 172.21.15.40:0/2152257348 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1800fd00 con 0x7f6d08003f70 2023-10-16T08:39:32.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.611+0000 7f6d233b8640 1 -- 172.21.15.40:0/2152257348 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d1c1f4450 con 0x7f6d08003f70 2023-10-16T08:39:32.635 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.611+0000 7f6d11ffb640 1 -- 172.21.15.40:0/2152257348 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d18015650 con 0x7f6d08003f70 2023-10-16T08:39:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.611+0000 7f6d233b8640 1 -- 172.21.15.40:0/2152257348 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d1c1f0920 con 0x7f6d08003f70 2023-10-16T08:39:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.611+0000 7f6d233b8640 1 -- 172.21.15.40:0/2152257348 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d1c1f0b40 con 0x7f6d08003f70 2023-10-16T08:39:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.611+0000 7f6d11ffb640 1 -- 172.21.15.40:0/2152257348 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6d180157b0 con 0x7f6d08003f70 2023-10-16T08:39:32.636 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.611+0000 7f6d11ffb640 1 --2- 172.21.15.40:0/2152257348 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f6d000664a0 0x7f6d00068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.611+0000 7f6d11ffb640 1 -- 172.21.15.40:0/2152257348 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6d18087110 con 0x7f6d08003f70 2023-10-16T08:39:32.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.611+0000 7f6d20953640 1 --2- 172.21.15.40:0/2152257348 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f6d000664a0 0x7f6d00068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.637 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.611+0000 7f6d20953640 1 --2- 172.21.15.40:0/2152257348 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f6d000664a0 0x7f6d00068960 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c0af000 tx=0x7f6d0c0046d0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:32.638 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.615+0000 7f6d11ffb640 1 -- 172.21.15.40:0/2152257348 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f6d18054590 con 0x7f6d08003f70 2023-10-16T08:39:32.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.767+0000 7f6d233b8640 1 -- 172.21.15.40:0/2152257348 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6d1c0ce690 con 0x7f6d08003f70 2023-10-16T08:39:32.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.767+0000 7f6d11ffb640 1 -- 172.21.15.40:0/2152257348 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6d18057bb0 con 0x7f6d08003f70 2023-10-16T08:39:32.774 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-16T08:39:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.771+0000 7f6d233b8640 1 -- 172.21.15.40:0/2152257348 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f6d000664a0 msgr2=0x7f6d00068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.771+0000 7f6d233b8640 1 --2- 172.21.15.40:0/2152257348 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f6d000664a0 0x7f6d00068960 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f6d0c0af000 tx=0x7f6d0c0046d0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.771+0000 7f6d233b8640 1 -- 172.21.15.40:0/2152257348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d08003f70 msgr2=0x7f6d1c1f3f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:32.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.771+0000 7f6d233b8640 1 --2- 172.21.15.40:0/2152257348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d08003f70 0x7f6d1c1f3f10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6d18011840 tx=0x7f6d18011d00 comp rx=0 tx=0).stop 2023-10-16T08:39:32.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.771+0000 7f6d233b8640 1 -- 172.21.15.40:0/2152257348 shutdown_connections 2023-10-16T08:39:32.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.771+0000 7f6d233b8640 1 --2- 172.21.15.40:0/2152257348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d1c151110 0x7f6d1c1f11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.771+0000 7f6d233b8640 1 --2- 172.21.15.40:0/2152257348 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6d1c150520 0x7f6d1c14ed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.771+0000 7f6d233b8640 1 --2- 172.21.15.40:0/2152257348 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f6d000664a0 0x7f6d00068960 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.771+0000 7f6d233b8640 1 --2- 172.21.15.40:0/2152257348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6d08003f70 0x7f6d1c1f3f10 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.771+0000 7f6d233b8640 1 -- 172.21.15.40:0/2152257348 >> 172.21.15.40:0/2152257348 conn(0x7f6d1c0ce890 msgr2=0x7f6d1c070e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.771+0000 7f6d233b8640 1 -- 172.21.15.40:0/2152257348 shutdown_connections 2023-10-16T08:39:32.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.771+0000 7f6d233b8640 1 -- 172.21.15.40:0/2152257348 wait complete. 2023-10-16T08:39:32.791 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:39:32.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.915+0000 7fe161360640 1 Processor -- start 2023-10-16T08:39:32.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.915+0000 7fe161360640 1 -- start start 2023-10-16T08:39:32.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.915+0000 7fe161360640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe15c0cae10 0x7fe15c0b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.915+0000 7fe161360640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe15c0adeb0 0x7fe15c15a500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.915+0000 7fe161360640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe15c0c98b0 con 0x7fe15c0adad0 2023-10-16T08:39:32.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.915+0000 7fe161360640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15c0b47c0 con 0x7fe15c0cae10 2023-10-16T08:39:32.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.915+0000 7fe161360640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c0c8f80 con 0x7fe15c0adeb0 2023-10-16T08:39:32.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.915+0000 7fe15b7fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe15c0adeb0 0x7fe15c15a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.915+0000 7fe15a7fc640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe15c0cae10 0x7fe15c0b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.915+0000 7fe15a7fc640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fe15c0cae10 0x7fe15c0b41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.40:45964/0 (socket says 172.21.15.40:45964) 2023-10-16T08:39:32.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.915+0000 7fe15a7fc640 1 -- 172.21.15.40:0/4093387180 learned_addr learned my addr 172.21.15.40:0/4093387180 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:32.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.915+0000 7fe15a7fc640 1 -- 172.21.15.40:0/4093387180 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe15c0adeb0 msgr2=0x7fe15c15a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:32.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.915+0000 7fe15a7fc640 1 --2- 172.21.15.40:0/4093387180 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe15c0adeb0 0x7fe15c15a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.915+0000 7fe15a7fc640 1 -- 172.21.15.40:0/4093387180 >> v1:172.21.15.86:6789/0 conn(0x7fe15c0adad0 legacy=0x7fe15c0ca700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:32.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.915+0000 7fe15a7fc640 1 -- 172.21.15.40:0/4093387180 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c0c9260 con 0x7fe15c0cae10 2023-10-16T08:39:32.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.919+0000 7fe15a7fc640 1 --2- 172.21.15.40:0/4093387180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe15c0cae10 0x7fe15c0b41e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe150009870 tx=0x7fe15001c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d320849cab45d3ca server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:32.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.919+0000 7fe159ffb640 1 -- 172.21.15.40:0/4093387180 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1500a95f0 con 0x7fe15c0cae10 2023-10-16T08:39:32.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.919+0000 7fe159ffb640 1 -- 172.21.15.40:0/4093387180 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1500a9750 con 0x7fe15c0cae10 2023-10-16T08:39:32.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.919+0000 7fe159ffb640 1 -- 172.21.15.40:0/4093387180 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1500a9ae0 con 0x7fe15c0cae10 2023-10-16T08:39:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.919+0000 7fe161360640 1 -- 172.21.15.40:0/4093387180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe15c0cae10 msgr2=0x7fe15c0b41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.919+0000 7fe161360640 1 --2- 172.21.15.40:0/4093387180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe15c0cae10 0x7fe15c0b41e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fe150009870 tx=0x7fe15001c510 comp rx=0 tx=0).stop 2023-10-16T08:39:32.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.919+0000 7fe161360640 1 -- 172.21.15.40:0/4093387180 shutdown_connections 2023-10-16T08:39:32.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.919+0000 7fe161360640 1 --2- 172.21.15.40:0/4093387180 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe15c0adeb0 0x7fe15c15a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.919+0000 7fe161360640 1 --2- 172.21.15.40:0/4093387180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe15c0cae10 0x7fe15c0b41e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.919+0000 7fe161360640 1 -- 172.21.15.40:0/4093387180 >> 172.21.15.40:0/4093387180 conn(0x7fe15c0ce890 msgr2=0x7fe15c0ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:32.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.919+0000 7fe161360640 1 -- 172.21.15.40:0/4093387180 shutdown_connections 2023-10-16T08:39:32.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.919+0000 7fe161360640 1 -- 172.21.15.40:0/4093387180 wait complete. 2023-10-16T08:39:32.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.919+0000 7fe161360640 1 Processor -- start 2023-10-16T08:39:32.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.919+0000 7fe161360640 1 -- start start 2023-10-16T08:39:32.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe161360640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe15c0adad0 0x7fe15c0c4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe161360640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe15c0adeb0 0x7fe15c0c6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe161360640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe15c0cae10 0x7fe15c13f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.932 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe161360640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c14de60 con 0x7fe15c0cae10 2023-10-16T08:39:32.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe161360640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c0c2730 con 0x7fe15c0adeb0 2023-10-16T08:39:32.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe161360640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c0c6b50 con 0x7fe15c0adad0 2023-10-16T08:39:32.933 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe15b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe15c0cae10 0x7fe15c13f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe15b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe15c0cae10 0x7fe15c13f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49810/0 (socket says 172.21.15.40:49810) 2023-10-16T08:39:32.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe15b7fe640 1 -- 172.21.15.40:0/3272852510 learned_addr learned my addr 172.21.15.40:0/3272852510 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:32.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe15affd640 1 --2- 172.21.15.40:0/3272852510 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe15c0adad0 0x7fe15c0c4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.934 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe15a7fc640 1 --2- 172.21.15.40:0/3272852510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe15c0adeb0 0x7fe15c0c6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe15b7fe640 1 -- 172.21.15.40:0/3272852510 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe15c0adad0 msgr2=0x7fe15c0c4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe15b7fe640 1 --2- 172.21.15.40:0/3272852510 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe15c0adad0 0x7fe15c0c4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe15b7fe640 1 -- 172.21.15.40:0/3272852510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe15c0adeb0 msgr2=0x7fe15c0c6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:32.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe15b7fe640 1 --2- 172.21.15.40:0/3272852510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe15c0adeb0 0x7fe15c0c6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:32.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe15b7fe640 1 -- 172.21.15.40:0/3272852510 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1500094d0 con 0x7fe15c0cae10 2023-10-16T08:39:32.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe15affd640 1 --2- 172.21.15.40:0/3272852510 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe15c0adad0 0x7fe15c0c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:32.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe15b7fe640 1 --2- 172.21.15.40:0/3272852510 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe15c0cae10 0x7fe15c13f780 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe15400ef90 tx=0x7fe15400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:32.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe1437fe640 1 -- 172.21.15.40:0/3272852510 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154009bf0 con 0x7fe15c0cae10 2023-10-16T08:39:32.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe1437fe640 1 -- 172.21.15.40:0/3272852510 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe15400f040 con 0x7fe15c0cae10 2023-10-16T08:39:32.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe1437fe640 1 -- 172.21.15.40:0/3272852510 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1540146c0 con 0x7fe15c0cae10 2023-10-16T08:39:32.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe161360640 1 -- 172.21.15.40:0/3272852510 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15c15b250 con 0x7fe15c0cae10 2023-10-16T08:39:32.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe161360640 1 -- 172.21.15.40:0/3272852510 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15c0c7110 con 0x7fe15c0cae10 2023-10-16T08:39:32.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe161360640 1 -- 172.21.15.40:0/3272852510 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15c067a20 con 0x7fe15c0cae10 2023-10-16T08:39:32.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe1437fe640 1 -- 172.21.15.40:0/3272852510 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe154014820 con 0x7fe15c0cae10 2023-10-16T08:39:32.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe1437fe640 1 --2- 172.21.15.40:0/3272852510 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe1480664a0 0x7fe148068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:32.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.923+0000 7fe1437fe640 1 -- 172.21.15.40:0/3272852510 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fe154084cb0 con 0x7fe15c0cae10 2023-10-16T08:39:32.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.927+0000 7fe15affd640 1 --2- 172.21.15.40:0/3272852510 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe1480664a0 0x7fe148068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:32.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.927+0000 7fe15affd640 1 --2- 172.21.15.40:0/3272852510 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe1480664a0 0x7fe148068960 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe14c000f70 tx=0x7fe14c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:32.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:32.931+0000 7fe1437fe640 1 -- 172.21.15.40:0/3272852510 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fe1540520a0 con 0x7fe15c0cae10 2023-10-16T08:39:33.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.091+0000 7fe161360640 1 -- 172.21.15.40:0/3272852510 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe15c0c28a0 con 0x7fe15c0cae10 2023-10-16T08:39:33.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.091+0000 7fe1437fe640 1 -- 172.21.15.40:0/3272852510 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe154018020 con 0x7fe15c0cae10 2023-10-16T08:39:33.097 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-16T08:39:33.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.095+0000 7fe161360640 1 -- 172.21.15.40:0/3272852510 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe1480664a0 msgr2=0x7fe148068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:33.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.095+0000 7fe161360640 1 --2- 172.21.15.40:0/3272852510 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe1480664a0 0x7fe148068960 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fe14c000f70 tx=0x7fe14c009040 comp rx=0 tx=0).stop 2023-10-16T08:39:33.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.095+0000 7fe161360640 1 -- 172.21.15.40:0/3272852510 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe15c0cae10 msgr2=0x7fe15c13f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:33.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.095+0000 7fe161360640 1 --2- 172.21.15.40:0/3272852510 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe15c0cae10 0x7fe15c13f780 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fe15400ef90 tx=0x7fe15400c490 comp rx=0 tx=0).stop 2023-10-16T08:39:33.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.095+0000 7fe161360640 1 -- 172.21.15.40:0/3272852510 shutdown_connections 2023-10-16T08:39:33.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.095+0000 7fe161360640 1 --2- 172.21.15.40:0/3272852510 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe1480664a0 0x7fe148068960 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.095+0000 7fe161360640 1 --2- 172.21.15.40:0/3272852510 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe15c0cae10 0x7fe15c13f780 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.095+0000 7fe161360640 1 --2- 172.21.15.40:0/3272852510 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe15c0adeb0 0x7fe15c0c6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.095+0000 7fe161360640 1 --2- 172.21.15.40:0/3272852510 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe15c0adad0 0x7fe15c0c4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.095+0000 7fe161360640 1 -- 172.21.15.40:0/3272852510 >> 172.21.15.40:0/3272852510 conn(0x7fe15c0ce890 msgr2=0x7fe15c0709c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:33.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.095+0000 7fe161360640 1 -- 172.21.15.40:0/3272852510 shutdown_connections 2023-10-16T08:39:33.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.095+0000 7fe161360640 1 -- 172.21.15.40:0/3272852510 wait complete. 2023-10-16T08:39:33.117 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:39:33.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.239+0000 7f98d545f640 1 Processor -- start 2023-10-16T08:39:33.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.239+0000 7f98d545f640 1 -- start start 2023-10-16T08:39:33.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.243+0000 7f98d545f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98d015ad80 0x7f98d006d660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:33.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.243+0000 7f98d545f640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f98d006dba0 0x7f98d00ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:33.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.243+0000 7f98d545f640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98d00cffd0 con 0x7f98d00ceb00 2023-10-16T08:39:33.249 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.243+0000 7f98d545f640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98d006dfe0 con 0x7f98d006dba0 2023-10-16T08:39:33.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.243+0000 7f98d545f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d00cf310 con 0x7f98d015ad80 2023-10-16T08:39:33.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.243+0000 7f98cf7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f98d00ceb00 0x7f98d00cc3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37028/0 (socket says 172.21.15.40:37028) 2023-10-16T08:39:33.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.243+0000 7f98cf7fe640 1 -- 172.21.15.40:0/1354970985 learned_addr learned my addr 172.21.15.40:0/1354970985 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:33.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.243+0000 7f98ce7fc640 1 --2- 172.21.15.40:0/1354970985 >> v2:172.21.15.40:3300/0 conn(0x7f98d006dba0 0x7f98d00ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:33.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.243+0000 7f98cdffb640 1 -- 172.21.15.40:0/1354970985 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1597773725 0 0) 0x7f98d00cffd0 con 0x7f98d00ceb00 2023-10-16T08:39:33.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.243+0000 7f98cdffb640 1 -- 172.21.15.40:0/1354970985 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98b8003530 con 0x7f98d00ceb00 2023-10-16T08:39:33.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.243+0000 7f98ceffd640 1 --2- 172.21.15.40:0/1354970985 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98d015ad80 0x7f98d006d660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:33.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.243+0000 7f98ce7fc640 1 -- 172.21.15.40:0/1354970985 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98d015ad80 msgr2=0x7f98d006d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:33.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.243+0000 7f98ce7fc640 1 --2- 172.21.15.40:0/1354970985 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98d015ad80 0x7f98d006d660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.243+0000 7f98ce7fc640 1 -- 172.21.15.40:0/1354970985 >> v1:172.21.15.40:6789/0 conn(0x7f98d00ceb00 legacy=0x7f98d00cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.243+0000 7f98ce7fc640 1 -- 172.21.15.40:0/1354970985 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98d00cf5f0 con 0x7f98d006dba0 2023-10-16T08:39:33.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.243+0000 7f98ceffd640 1 --2- 172.21.15.40:0/1354970985 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98d015ad80 0x7f98d006d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:33.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.243+0000 7f98ce7fc640 1 --2- 172.21.15.40:0/1354970985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d006dba0 0x7f98d00ce5c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f98c00097c0 tx=0x7f98c001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89f758faf40a829e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:33.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.243+0000 7f98cdffb640 1 -- 172.21.15.40:0/1354970985 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c00ad660 con 0x7f98d006dba0 2023-10-16T08:39:33.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.243+0000 7f98cdffb640 1 -- 172.21.15.40:0/1354970985 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c00ad7c0 con 0x7f98d006dba0 2023-10-16T08:39:33.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.243+0000 7f98cdffb640 1 -- 172.21.15.40:0/1354970985 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c00adb50 con 0x7f98d006dba0 2023-10-16T08:39:33.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98d545f640 1 -- 172.21.15.40:0/1354970985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d006dba0 msgr2=0x7f98d00ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:33.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98d545f640 1 --2- 172.21.15.40:0/1354970985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d006dba0 0x7f98d00ce5c0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f98c00097c0 tx=0x7f98c001c450 comp rx=0 tx=0).stop 2023-10-16T08:39:33.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98d545f640 1 -- 172.21.15.40:0/1354970985 shutdown_connections 2023-10-16T08:39:33.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98d545f640 1 --2- 172.21.15.40:0/1354970985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d006dba0 0x7f98d00ce5c0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98d545f640 1 --2- 172.21.15.40:0/1354970985 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98d015ad80 0x7f98d006d660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98d545f640 1 -- 172.21.15.40:0/1354970985 >> 172.21.15.40:0/1354970985 conn(0x7f98d0071580 msgr2=0x7f98d0071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:33.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98d545f640 1 -- 172.21.15.40:0/1354970985 shutdown_connections 2023-10-16T08:39:33.259 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98d545f640 1 -- 172.21.15.40:0/1354970985 wait complete. 2023-10-16T08:39:33.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98d545f640 1 Processor -- start 2023-10-16T08:39:33.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98d545f640 1 -- start start 2023-10-16T08:39:33.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98d545f640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d006dba0 0x7f98d014ac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:33.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98d545f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d00ceb00 0x7f98d0149020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:33.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98d545f640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98d015ad80 0x7f98d00c4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:33.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98d545f640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d0149560 con 0x7f98d006dba0 2023-10-16T08:39:33.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98d545f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98d00c7380 con 0x7f98d00ceb00 2023-10-16T08:39:33.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98d545f640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98d0159a50 con 0x7f98d015ad80 2023-10-16T08:39:33.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98ceffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d006dba0 0x7f98d014ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:33.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98ceffd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d006dba0 0x7f98d014ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49834/0 (socket says 172.21.15.40:49834) 2023-10-16T08:39:33.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98ceffd640 1 -- 172.21.15.40:0/278729021 learned_addr learned my addr 172.21.15.40:0/278729021 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:33.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98cf7fe640 1 --2- 172.21.15.40:0/278729021 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98d015ad80 0x7f98d00c4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:33.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98ce7fc640 1 --2- 172.21.15.40:0/278729021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d00ceb00 0x7f98d0149020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:33.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98ceffd640 1 -- 172.21.15.40:0/278729021 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98d015ad80 msgr2=0x7f98d00c4ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:33.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98ceffd640 1 --2- 172.21.15.40:0/278729021 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98d015ad80 0x7f98d00c4ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98ceffd640 1 -- 172.21.15.40:0/278729021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d00ceb00 msgr2=0x7f98d0149020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:33.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98ceffd640 1 --2- 172.21.15.40:0/278729021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d00ceb00 0x7f98d0149020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98ceffd640 1 -- 172.21.15.40:0/278729021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98c0009420 con 0x7f98d006dba0 2023-10-16T08:39:33.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98ce7fc640 1 --2- 172.21.15.40:0/278729021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d00ceb00 0x7f98d0149020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:33.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98ceffd640 1 --2- 172.21.15.40:0/278729021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d006dba0 0x7f98d014ac00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f98c400c820 tx=0x7f98c400cce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:33.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98b77fe640 1 -- 172.21.15.40:0/278729021 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c4007a30 con 0x7f98d006dba0 2023-10-16T08:39:33.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98b77fe640 1 -- 172.21.15.40:0/278729021 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98c4007b90 con 0x7f98d006dba0 2023-10-16T08:39:33.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98b77fe640 1 -- 172.21.15.40:0/278729021 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98c400ce10 con 0x7f98d006dba0 2023-10-16T08:39:33.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98d545f640 1 -- 172.21.15.40:0/278729021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98d00c6e40 con 0x7f98d006dba0 2023-10-16T08:39:33.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.247+0000 7f98d545f640 1 -- 172.21.15.40:0/278729021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98d0159440 con 0x7f98d006dba0 2023-10-16T08:39:33.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.251+0000 7f98d545f640 1 -- 172.21.15.40:0/278729021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98d01592a0 con 0x7f98d006dba0 2023-10-16T08:39:33.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.251+0000 7f98b77fe640 1 -- 172.21.15.40:0/278729021 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f98c40113f0 con 0x7f98d006dba0 2023-10-16T08:39:33.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.251+0000 7f98b77fe640 1 --2- 172.21.15.40:0/278729021 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f98bc0664a0 0x7f98bc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:33.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.251+0000 7f98b77fe640 1 -- 172.21.15.40:0/278729021 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f98c4084070 con 0x7f98d006dba0 2023-10-16T08:39:33.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.251+0000 7f98ce7fc640 1 --2- 172.21.15.40:0/278729021 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f98bc0664a0 0x7f98bc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:33.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.251+0000 7f98ce7fc640 1 --2- 172.21.15.40:0/278729021 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f98bc0664a0 0x7f98bc068960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f98c00af000 tx=0x7f98c0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:33.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.255+0000 7f98b77fe640 1 -- 172.21.15.40:0/278729021 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f98d01592a0 con 0x7f98d006dba0 2023-10-16T08:39:33.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.411+0000 7f98d545f640 1 -- 172.21.15.40:0/278729021 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f98d01592a0 con 0x7f98d006dba0 2023-10-16T08:39:33.417 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.411+0000 7f98b77fe640 1 -- 172.21.15.40:0/278729021 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7f98d01592a0 con 0x7f98d006dba0 2023-10-16T08:39:33.417 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-16T08:39:33.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.415+0000 7f98d545f640 1 -- 172.21.15.40:0/278729021 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f98bc0664a0 msgr2=0x7f98bc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:33.419 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.415+0000 7f98d545f640 1 --2- 172.21.15.40:0/278729021 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f98bc0664a0 0x7f98bc068960 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f98c00af000 tx=0x7f98c0008040 comp rx=0 tx=0).stop 2023-10-16T08:39:33.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.415+0000 7f98d545f640 1 -- 172.21.15.40:0/278729021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d006dba0 msgr2=0x7f98d014ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:33.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.415+0000 7f98d545f640 1 --2- 172.21.15.40:0/278729021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d006dba0 0x7f98d014ac00 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f98c400c820 tx=0x7f98c400cce0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.415+0000 7f98d545f640 1 -- 172.21.15.40:0/278729021 shutdown_connections 2023-10-16T08:39:33.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.415+0000 7f98d545f640 1 --2- 172.21.15.40:0/278729021 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f98d015ad80 0x7f98d00c4ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.415+0000 7f98d545f640 1 --2- 172.21.15.40:0/278729021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f98d00ceb00 0x7f98d0149020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.415+0000 7f98d545f640 1 --2- 172.21.15.40:0/278729021 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f98bc0664a0 0x7f98bc068960 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.415+0000 7f98d545f640 1 --2- 172.21.15.40:0/278729021 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f98d006dba0 0x7f98d014ac00 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.415+0000 7f98d545f640 1 -- 172.21.15.40:0/278729021 >> 172.21.15.40:0/278729021 conn(0x7f98d0071580 msgr2=0x7f98d00b14d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:33.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.415+0000 7f98d545f640 1 -- 172.21.15.40:0/278729021 shutdown_connections 2023-10-16T08:39:33.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.415+0000 7f98d545f640 1 -- 172.21.15.40:0/278729021 wait complete. 2023-10-16T08:39:33.437 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:39:33.565 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.559+0000 7f994c9d2640 1 Processor -- start 2023-10-16T08:39:33.566 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.559+0000 7f994c9d2640 1 -- start start 2023-10-16T08:39:33.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.559+0000 7f994c9d2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f99480d15c0 0x7f99480cf060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:33.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.559+0000 7f994c9d2640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99480b1ec0 con 0x7f99480af740 2023-10-16T08:39:33.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.559+0000 7f994c9d2640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99480b1340 con 0x7f99480afb20 2023-10-16T08:39:33.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.559+0000 7f994c9d2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99480b24c0 con 0x7f99480d15c0 2023-10-16T08:39:33.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.563+0000 7f993ffff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f99480d15c0 0x7f99480cf060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:33.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.563+0000 7f994659c640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f99480af740 0x7f99480b0b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:37030/0 (socket says 172.21.15.40:37030) 2023-10-16T08:39:33.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.563+0000 7f994659c640 1 -- 172.21.15.40:0/4171814574 learned_addr learned my addr 172.21.15.40:0/4171814574 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:33.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.563+0000 7f993ffff640 1 -- 172.21.15.40:0/4171814574 >> v1:172.21.15.86:6789/0 conn(0x7f99480afb20 legacy=0x7f99480adb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:33.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.563+0000 7f993ffff640 1 -- 172.21.15.40:0/4171814574 >> v1:172.21.15.40:6789/0 conn(0x7f99480af740 legacy=0x7f99480b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:33.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.563+0000 7f993ffff640 1 -- 172.21.15.40:0/4171814574 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99480b1620 con 0x7f99480d15c0 2023-10-16T08:39:33.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.563+0000 7f9945d9b640 1 -- 172.21.15.40:0/4171814574 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 289769725 0 0) 0x7f99480b1ec0 con 0x7f99480af740 2023-10-16T08:39:33.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.563+0000 7f993ffff640 1 --2- 172.21.15.40:0/4171814574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99480d15c0 0x7f99480cf060 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f99300097c0 tx=0x7f993001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32b1e176adcb870d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:33.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.563+0000 7f9945d9b640 1 -- 172.21.15.40:0/4171814574 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99300ab650 con 0x7f99480d15c0 2023-10-16T08:39:33.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.563+0000 7f9945d9b640 1 -- 172.21.15.40:0/4171814574 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99300ab7b0 con 0x7f99480d15c0 2023-10-16T08:39:33.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.563+0000 7f9945d9b640 1 -- 172.21.15.40:0/4171814574 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99300abb40 con 0x7f99480d15c0 2023-10-16T08:39:33.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.563+0000 7f994c9d2640 1 -- 172.21.15.40:0/4171814574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99480d15c0 msgr2=0x7f99480cf060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:33.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.563+0000 7f994c9d2640 1 --2- 172.21.15.40:0/4171814574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99480d15c0 0x7f99480cf060 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f99300097c0 tx=0x7f993001c450 comp rx=0 tx=0).stop 2023-10-16T08:39:33.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.563+0000 7f994c9d2640 1 -- 172.21.15.40:0/4171814574 shutdown_connections 2023-10-16T08:39:33.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.563+0000 7f994c9d2640 1 --2- 172.21.15.40:0/4171814574 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99480d15c0 0x7f99480cf060 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.563+0000 7f994c9d2640 1 -- 172.21.15.40:0/4171814574 >> 172.21.15.40:0/4171814574 conn(0x7f9948099fa0 msgr2=0x7f99480d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:33.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.563+0000 7f994c9d2640 1 -- 172.21.15.40:0/4171814574 shutdown_connections 2023-10-16T08:39:33.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.563+0000 7f994c9d2640 1 -- 172.21.15.40:0/4171814574 wait complete. 2023-10-16T08:39:33.574 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f994c9d2640 1 Processor -- start 2023-10-16T08:39:33.575 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f994c9d2640 1 -- start start 2023-10-16T08:39:33.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f994c9d2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99480af740 0x7f9948150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:33.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f994c9d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99480afb20 0x7f9948157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:33.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f994c9d2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99480d15c0 0x7f994806b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:33.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f994c9d2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9948150db0 con 0x7f99480af740 2023-10-16T08:39:33.578 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f994c9d2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9948158390 con 0x7f99480afb20 2023-10-16T08:39:33.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f994c9d2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99481f8fa0 con 0x7f99480d15c0 2023-10-16T08:39:33.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f994659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99480af740 0x7f9948150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:33.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f994659c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99480af740 0x7f9948150870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49856/0 (socket says 172.21.15.40:49856) 2023-10-16T08:39:33.579 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f994659c640 1 -- 172.21.15.40:0/4275909360 learned_addr learned my addr 172.21.15.40:0/4275909360 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:33.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f993ffff640 1 --2- 172.21.15.40:0/4275909360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99480afb20 0x7f9948157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:33.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f9946d9d640 1 --2- 172.21.15.40:0/4275909360 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99480d15c0 0x7f994806b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:33.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f994659c640 1 -- 172.21.15.40:0/4275909360 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99480d15c0 msgr2=0x7f994806b660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:33.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f994659c640 1 --2- 172.21.15.40:0/4275909360 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99480d15c0 0x7f994806b660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f994659c640 1 -- 172.21.15.40:0/4275909360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99480afb20 msgr2=0x7f9948157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:33.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f994659c640 1 --2- 172.21.15.40:0/4275909360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99480afb20 0x7f9948157e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f994659c640 1 -- 172.21.15.40:0/4275909360 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9930009420 con 0x7f99480af740 2023-10-16T08:39:33.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f994659c640 1 --2- 172.21.15.40:0/4275909360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99480af740 0x7f9948150870 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f993800c4a0 tx=0x7f993800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:33.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f993effd640 1 -- 172.21.15.40:0/4275909360 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99380074e0 con 0x7f99480af740 2023-10-16T08:39:33.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f993effd640 1 -- 172.21.15.40:0/4275909360 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9938007640 con 0x7f99480af740 2023-10-16T08:39:33.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f993effd640 1 -- 172.21.15.40:0/4275909360 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9938004a30 con 0x7f99480af740 2023-10-16T08:39:33.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f994c9d2640 1 -- 172.21.15.40:0/4275909360 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99481f9280 con 0x7f99480af740 2023-10-16T08:39:33.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f994c9d2640 1 -- 172.21.15.40:0/4275909360 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9948148a70 con 0x7f99480af740 2023-10-16T08:39:33.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f994c9d2640 1 -- 172.21.15.40:0/4275909360 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9948148c60 con 0x7f99480af740 2023-10-16T08:39:33.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.567+0000 7f993effd640 1 -- 172.21.15.40:0/4275909360 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f9938004b90 con 0x7f99480af740 2023-10-16T08:39:33.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.571+0000 7f993effd640 1 --2- 172.21.15.40:0/4275909360 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f99280664a0 0x7f9928068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:33.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.571+0000 7f993effd640 1 -- 172.21.15.40:0/4275909360 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f99380831a0 con 0x7f99480af740 2023-10-16T08:39:33.584 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.571+0000 7f993ffff640 1 --2- 172.21.15.40:0/4275909360 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f99280664a0 0x7f9928068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:33.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.571+0000 7f993ffff640 1 --2- 172.21.15.40:0/4275909360 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f99280664a0 0x7f9928068960 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f99300af000 tx=0x7f99300a7000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:33.585 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.575+0000 7f993effd640 1 -- 172.21.15.40:0/4275909360 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f9938050590 con 0x7f99480af740 2023-10-16T08:39:33.753 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.747+0000 7f994c9d2640 1 -- 172.21.15.40:0/4275909360 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f994804e9c0 con 0x7f99480af740 2023-10-16T08:39:33.754 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.747+0000 7f993effd640 1 -- 172.21.15.40:0/4275909360 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7f9938018080 con 0x7f99480af740 2023-10-16T08:39:33.755 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-16T08:39:33.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.751+0000 7f994c9d2640 1 -- 172.21.15.40:0/4275909360 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f99280664a0 msgr2=0x7f9928068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:33.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.751+0000 7f994c9d2640 1 --2- 172.21.15.40:0/4275909360 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f99280664a0 0x7f9928068960 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f99300af000 tx=0x7f99300a7000 comp rx=0 tx=0).stop 2023-10-16T08:39:33.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.751+0000 7f994c9d2640 1 -- 172.21.15.40:0/4275909360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99480af740 msgr2=0x7f9948150870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:33.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.751+0000 7f994c9d2640 1 --2- 172.21.15.40:0/4275909360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99480af740 0x7f9948150870 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f993800c4a0 tx=0x7f993800c960 comp rx=0 tx=0).stop 2023-10-16T08:39:33.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.751+0000 7f994c9d2640 1 -- 172.21.15.40:0/4275909360 shutdown_connections 2023-10-16T08:39:33.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.751+0000 7f994c9d2640 1 --2- 172.21.15.40:0/4275909360 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f99480d15c0 0x7f994806b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.751+0000 7f994c9d2640 1 --2- 172.21.15.40:0/4275909360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f99480afb20 0x7f9948157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.751+0000 7f994c9d2640 1 --2- 172.21.15.40:0/4275909360 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f99280664a0 0x7f9928068960 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.751+0000 7f994c9d2640 1 --2- 172.21.15.40:0/4275909360 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f99480af740 0x7f9948150870 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.751+0000 7f994c9d2640 1 -- 172.21.15.40:0/4275909360 >> 172.21.15.40:0/4275909360 conn(0x7f9948099fa0 msgr2=0x7f99480cb580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:33.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.751+0000 7f994c9d2640 1 -- 172.21.15.40:0/4275909360 shutdown_connections 2023-10-16T08:39:33.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.751+0000 7f994c9d2640 1 -- 172.21.15.40:0/4275909360 wait complete. 2023-10-16T08:39:33.774 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2023-10-16T08:39:33.774 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2023-10-16T08:39:33.781 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:39:33.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.899+0000 7f58c22a4640 1 Processor -- start 2023-10-16T08:39:33.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.899+0000 7f58c22a4640 1 -- start start 2023-10-16T08:39:33.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.903+0000 7f58c22a4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58bc15d180 0x7f58bc06b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:33.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.903+0000 7f58c22a4640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f58bc0ad820 0x7f58bc0c9c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:33.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.903+0000 7f58c22a4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58bc0cf480 con 0x7f58bc15d560 2023-10-16T08:39:33.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.903+0000 7f58c22a4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58bc0cf760 con 0x7f58bc15d180 2023-10-16T08:39:33.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.903+0000 7f58c22a4640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58bc0c8fb0 con 0x7f58bc0ad820 2023-10-16T08:39:33.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.903+0000 7f58bbfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58bc15d180 0x7f58bc06b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:33.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.903+0000 7f58bbfff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f58bc15d180 0x7f58bc06b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49870/0 (socket says 172.21.15.40:49870) 2023-10-16T08:39:33.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.903+0000 7f58bbfff640 1 -- 172.21.15.40:0/1368774369 learned_addr learned my addr 172.21.15.40:0/1368774369 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:33.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.903+0000 7f58c0841640 1 --2- 172.21.15.40:0/1368774369 >> v2:172.21.15.86:3300/0 conn(0x7f58bc0ad820 0x7f58bc0c9c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:33.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.903+0000 7f58bbfff640 1 -- 172.21.15.40:0/1368774369 >> v2:172.21.15.86:3300/0 conn(0x7f58bc0ad820 msgr2=0x7f58bc0c9c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:33.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.903+0000 7f58bbfff640 1 --2- 172.21.15.40:0/1368774369 >> v2:172.21.15.86:3300/0 conn(0x7f58bc0ad820 0x7f58bc0c9c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.903+0000 7f58bbfff640 1 -- 172.21.15.40:0/1368774369 >> v1:172.21.15.40:6789/0 conn(0x7f58bc15d560 legacy=0x7f58bc0c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:33.912 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.903+0000 7f58bbfff640 1 -- 172.21.15.40:0/1368774369 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58bc0c9290 con 0x7f58bc15d180 2023-10-16T08:39:33.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.903+0000 7f58bbfff640 1 --2- 172.21.15.40:0/1368774369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58bc15d180 0x7f58bc06b7e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0669b0 tx=0x7f58ac01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e760aa68d9287b2d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:33.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.903+0000 7f58baffd640 1 -- 172.21.15.40:0/1368774369 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0a58b0 con 0x7f58bc15d180 2023-10-16T08:39:33.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58baffd640 1 -- 172.21.15.40:0/1368774369 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ac0a5a10 con 0x7f58bc15d180 2023-10-16T08:39:33.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58baffd640 1 -- 172.21.15.40:0/1368774369 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ac0a5da0 con 0x7f58bc15d180 2023-10-16T08:39:33.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c22a4640 1 -- 172.21.15.40:0/1368774369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58bc15d180 msgr2=0x7f58bc06b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:33.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c22a4640 1 --2- 172.21.15.40:0/1368774369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58bc15d180 0x7f58bc06b7e0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f58bc0669b0 tx=0x7f58ac01c9d0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c22a4640 1 -- 172.21.15.40:0/1368774369 shutdown_connections 2023-10-16T08:39:33.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c22a4640 1 --2- 172.21.15.40:0/1368774369 >> v2:172.21.15.86:3300/0 conn(0x7f58bc0ad820 0x7f58bc0c9c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c22a4640 1 --2- 172.21.15.40:0/1368774369 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58bc15d180 0x7f58bc06b7e0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c22a4640 1 -- 172.21.15.40:0/1368774369 >> 172.21.15.40:0/1368774369 conn(0x7f58bc0ce960 msgr2=0x7f58bc0ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:33.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c22a4640 1 -- 172.21.15.40:0/1368774369 shutdown_connections 2023-10-16T08:39:33.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c22a4640 1 -- 172.21.15.40:0/1368774369 wait complete. 2023-10-16T08:39:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c22a4640 1 Processor -- start 2023-10-16T08:39:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c22a4640 1 -- start start 2023-10-16T08:39:33.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c22a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58bc0ad820 0x7f58bc157350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c22a4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58bc15d180 0x7f58bc0c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c22a4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58bc15d560 0x7f58bc0c36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:33.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c22a4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc157890 con 0x7f58bc15d560 2023-10-16T08:39:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c22a4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc0c2630 con 0x7f58bc0ad820 2023-10-16T08:39:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c22a4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58bc0c2910 con 0x7f58bc15d180 2023-10-16T08:39:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c0841640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58bc15d560 0x7f58bc0c36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:33.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c0841640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58bc15d560 0x7f58bc0c36d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49882/0 (socket says 172.21.15.40:49882) 2023-10-16T08:39:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c0841640 1 -- 172.21.15.40:0/1265592985 learned_addr learned my addr 172.21.15.40:0/1265592985 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58bbfff640 1 --2- 172.21.15.40:0/1265592985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58bc0ad820 0x7f58bc157350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58bb7fe640 1 --2- 172.21.15.40:0/1265592985 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58bc15d180 0x7f58bc0c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:33.923 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c0841640 1 -- 172.21.15.40:0/1265592985 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58bc15d180 msgr2=0x7f58bc0c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:33.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c0841640 1 --2- 172.21.15.40:0/1265592985 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58bc15d180 0x7f58bc0c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c0841640 1 -- 172.21.15.40:0/1265592985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58bc0ad820 msgr2=0x7f58bc157350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:33.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c0841640 1 --2- 172.21.15.40:0/1265592985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58bc0ad820 0x7f58bc157350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:33.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c0841640 1 -- 172.21.15.40:0/1265592985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ac009b30 con 0x7f58bc15d560 2023-10-16T08:39:33.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c0841640 1 --2- 172.21.15.40:0/1265592985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58bc15d560 0x7f58bc0c36d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f58b400d580 tx=0x7f58b400da40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:33.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58b8ff9640 1 -- 172.21.15.40:0/1265592985 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b40125f0 con 0x7f58bc15d560 2023-10-16T08:39:33.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58b8ff9640 1 -- 172.21.15.40:0/1265592985 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58b4012750 con 0x7f58bc15d560 2023-10-16T08:39:33.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c22a4640 1 -- 172.21.15.40:0/1265592985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58bc0c2bf0 con 0x7f58bc15d560 2023-10-16T08:39:33.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58b8ff9640 1 -- 172.21.15.40:0/1265592985 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58b4012b00 con 0x7f58bc15d560 2023-10-16T08:39:33.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.907+0000 7f58c22a4640 1 -- 172.21.15.40:0/1265592985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58bc0c31b0 con 0x7f58bc15d560 2023-10-16T08:39:33.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.911+0000 7f58c22a4640 1 -- 172.21.15.40:0/1265592985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58bc04e9c0 con 0x7f58bc15d560 2023-10-16T08:39:33.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.911+0000 7f58b8ff9640 1 -- 172.21.15.40:0/1265592985 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f58b4012c60 con 0x7f58bc15d560 2023-10-16T08:39:33.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.911+0000 7f58b8ff9640 1 --2- 172.21.15.40:0/1265592985 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f58a80664a0 0x7f58a8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:33.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.911+0000 7f58b8ff9640 1 -- 172.21.15.40:0/1265592985 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f58b4084390 con 0x7f58bc15d560 2023-10-16T08:39:33.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.911+0000 7f58bbfff640 1 --2- 172.21.15.40:0/1265592985 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f58a80664a0 0x7f58a8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:33.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.911+0000 7f58bbfff640 1 --2- 172.21.15.40:0/1265592985 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f58a80664a0 0x7f58a8068960 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0ae000 tx=0x7f58ac0a7070 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:33.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:33.915+0000 7f58b8ff9640 1 -- 172.21.15.40:0/1265592985 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f58b4051790 con 0x7f58bc15d560 2023-10-16T08:39:34.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.075+0000 7f58c22a4640 1 -- 172.21.15.40:0/1265592985 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f58bc0b48e0 con 0x7f58bc15d560 2023-10-16T08:39:34.081 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.075+0000 7f58b8ff9640 1 -- 172.21.15.40:0/1265592985 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16741 (secure 0 0 0) 0x7f58b4054db0 con 0x7f58bc15d560 2023-10-16T08:39:34.081 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:39:34.082 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:39:26.948454+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:39:34.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.079+0000 7f58c22a4640 1 -- 172.21.15.40:0/1265592985 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f58a80664a0 msgr2=0x7f58a8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:34.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.079+0000 7f58c22a4640 1 --2- 172.21.15.40:0/1265592985 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f58a80664a0 0x7f58a8068960 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f58ac0ae000 tx=0x7f58ac0a7070 comp rx=0 tx=0).stop 2023-10-16T08:39:34.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.079+0000 7f58c22a4640 1 -- 172.21.15.40:0/1265592985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58bc15d560 msgr2=0x7f58bc0c36d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:34.084 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.079+0000 7f58c22a4640 1 --2- 172.21.15.40:0/1265592985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58bc15d560 0x7f58bc0c36d0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f58b400d580 tx=0x7f58b400da40 comp rx=0 tx=0).stop 2023-10-16T08:39:34.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.079+0000 7f58c22a4640 1 -- 172.21.15.40:0/1265592985 shutdown_connections 2023-10-16T08:39:34.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.079+0000 7f58c22a4640 1 --2- 172.21.15.40:0/1265592985 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f58a80664a0 0x7f58a8068960 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.079+0000 7f58c22a4640 1 --2- 172.21.15.40:0/1265592985 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f58bc15d560 0x7f58bc0c36d0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.079+0000 7f58c22a4640 1 --2- 172.21.15.40:0/1265592985 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f58bc15d180 0x7f58bc0c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.079+0000 7f58c22a4640 1 --2- 172.21.15.40:0/1265592985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f58bc0ad820 0x7f58bc157350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.079+0000 7f58c22a4640 1 -- 172.21.15.40:0/1265592985 >> 172.21.15.40:0/1265592985 conn(0x7f58bc0ce960 msgr2=0x7f58bc0ac8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:34.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.079+0000 7f58c22a4640 1 -- 172.21.15.40:0/1265592985 shutdown_connections 2023-10-16T08:39:34.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.079+0000 7f58c22a4640 1 -- 172.21.15.40:0/1265592985 wait complete. 2023-10-16T08:39:34.100 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-16T08:38:33.014668+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-16T08:38:43.055215+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-10-16T08:38:43.971666+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-10-16T08:39:34.101 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-16T08:39:34.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.215+0000 7fac7bae2640 1 Processor -- start 2023-10-16T08:39:34.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.215+0000 7fac7bae2640 1 -- start start 2023-10-16T08:39:34.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.215+0000 7fac7bae2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fac7415cdc0 0x7fac740d1040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:34.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.215+0000 7fac7bae2640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fac740d3310 0x7fac740b5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:34.221 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.215+0000 7fac7bae2640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac740d1870 con 0x7fac740b8590 2023-10-16T08:39:34.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.215+0000 7fac7bae2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac740d1b50 con 0x7fac7415cdc0 2023-10-16T08:39:34.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.215+0000 7fac7bae2640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac740d1e30 con 0x7fac740d3310 2023-10-16T08:39:34.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.215+0000 7fac7987e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fac7415cdc0 0x7fac740d1040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:34.222 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.215+0000 7fac7987e640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fac7415cdc0 0x7fac740d1040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49890/0 (socket says 172.21.15.40:49890) 2023-10-16T08:39:34.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.215+0000 7fac7987e640 1 -- 172.21.15.40:0/2856011029 learned_addr learned my addr 172.21.15.40:0/2856011029 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:34.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.215+0000 7fac7987e640 1 -- 172.21.15.40:0/2856011029 >> v2:172.21.15.86:3300/0 conn(0x7fac740d3310 msgr2=0x7fac740b5da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:34.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.215+0000 7fac7987e640 1 --2- 172.21.15.40:0/2856011029 >> v2:172.21.15.86:3300/0 conn(0x7fac740d3310 0x7fac740b5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.215+0000 7fac7987e640 1 -- 172.21.15.40:0/2856011029 >> v1:172.21.15.86:6789/0 conn(0x7fac740b8590 legacy=0x7fac740d2c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:34.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.215+0000 7fac7987e640 1 -- 172.21.15.40:0/2856011029 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac740d2110 con 0x7fac7415cdc0 2023-10-16T08:39:34.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.215+0000 7fac7987e640 1 --2- 172.21.15.40:0/2856011029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac7415cdc0 0x7fac740d1040 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fac740d0f60 tx=0x7fac7001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f797bea2388ee5a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:34.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7887c640 1 -- 172.21.15.40:0/2856011029 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac700ab670 con 0x7fac7415cdc0 2023-10-16T08:39:34.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7887c640 1 -- 172.21.15.40:0/2856011029 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac700ab7d0 con 0x7fac7415cdc0 2023-10-16T08:39:34.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7887c640 1 -- 172.21.15.40:0/2856011029 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac700abb60 con 0x7fac7415cdc0 2023-10-16T08:39:34.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7bae2640 1 -- 172.21.15.40:0/2856011029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac7415cdc0 msgr2=0x7fac740d1040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:34.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7bae2640 1 --2- 172.21.15.40:0/2856011029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac7415cdc0 0x7fac740d1040 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fac740d0f60 tx=0x7fac7001c490 comp rx=0 tx=0).stop 2023-10-16T08:39:34.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7bae2640 1 -- 172.21.15.40:0/2856011029 shutdown_connections 2023-10-16T08:39:34.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7bae2640 1 --2- 172.21.15.40:0/2856011029 >> v2:172.21.15.86:3300/0 conn(0x7fac740d3310 0x7fac740b5da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7bae2640 1 --2- 172.21.15.40:0/2856011029 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac7415cdc0 0x7fac740d1040 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7bae2640 1 -- 172.21.15.40:0/2856011029 >> 172.21.15.40:0/2856011029 conn(0x7fac7406a290 msgr2=0x7fac7406a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:34.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7bae2640 1 -- 172.21.15.40:0/2856011029 shutdown_connections 2023-10-16T08:39:34.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7bae2640 1 -- 172.21.15.40:0/2856011029 wait complete. 2023-10-16T08:39:34.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7bae2640 1 Processor -- start 2023-10-16T08:39:34.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7bae2640 1 -- start start 2023-10-16T08:39:34.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7bae2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac740b8590 0x7fac741542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:34.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7bae2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac740d3310 0x7fac7414f570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:34.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7bae2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac7415cdc0 0x7fac7415f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:34.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7bae2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7414fee0 con 0x7fac740d3310 2023-10-16T08:39:34.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7bae2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac7414ed30 con 0x7fac7415cdc0 2023-10-16T08:39:34.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7bae2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac7415d320 con 0x7fac740b8590 2023-10-16T08:39:34.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7907d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac740d3310 0x7fac7414f570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:34.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7907d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac740d3310 0x7fac7414f570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49894/0 (socket says 172.21.15.40:49894) 2023-10-16T08:39:34.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7907d640 1 -- 172.21.15.40:0/3206831791 learned_addr learned my addr 172.21.15.40:0/3206831791 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:34.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7907d640 1 -- 172.21.15.40:0/3206831791 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac740b8590 msgr2=0x7fac741542f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:34.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7a07f640 1 --2- 172.21.15.40:0/3206831791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac7415cdc0 0x7fac7415f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:34.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7907d640 1 --2- 172.21.15.40:0/3206831791 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac740b8590 0x7fac741542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7907d640 1 -- 172.21.15.40:0/3206831791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac7415cdc0 msgr2=0x7fac7415f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:34.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7907d640 1 --2- 172.21.15.40:0/3206831791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac7415cdc0 0x7fac7415f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7907d640 1 -- 172.21.15.40:0/3206831791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac700094d0 con 0x7fac740d3310 2023-10-16T08:39:34.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7907d640 1 --2- 172.21.15.40:0/3206831791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac740d3310 0x7fac7414f570 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fac6400c490 tx=0x7fac64001e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:34.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac6a7fc640 1 -- 172.21.15.40:0/3206831791 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac64003600 con 0x7fac740d3310 2023-10-16T08:39:34.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac6a7fc640 1 -- 172.21.15.40:0/3206831791 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac64003780 con 0x7fac740d3310 2023-10-16T08:39:34.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7bae2640 1 -- 172.21.15.40:0/3206831791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac7414efd0 con 0x7fac740d3310 2023-10-16T08:39:34.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac6a7fc640 1 -- 172.21.15.40:0/3206831791 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac64003b70 con 0x7fac740d3310 2023-10-16T08:39:34.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.219+0000 7fac7bae2640 1 -- 172.21.15.40:0/3206831791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac7414a1b0 con 0x7fac740d3310 2023-10-16T08:39:34.236 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.223+0000 7fac7bae2640 1 -- 172.21.15.40:0/3206831791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac7404e9c0 con 0x7fac740d3310 2023-10-16T08:39:34.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.223+0000 7fac6a7fc640 1 -- 172.21.15.40:0/3206831791 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fac64003cd0 con 0x7fac740d3310 2023-10-16T08:39:34.237 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.223+0000 7fac6a7fc640 1 --2- 172.21.15.40:0/3206831791 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fac580664a0 0x7fac58068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:34.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.223+0000 7fac6a7fc640 1 -- 172.21.15.40:0/3206831791 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fac640834c0 con 0x7fac740d3310 2023-10-16T08:39:34.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.223+0000 7fac7987e640 1 --2- 172.21.15.40:0/3206831791 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fac580664a0 0x7fac58068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:34.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.223+0000 7fac7987e640 1 --2- 172.21.15.40:0/3206831791 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fac580664a0 0x7fac58068960 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fac700af000 tx=0x7fac70008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:34.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.227+0000 7fac6a7fc640 1 -- 172.21.15.40:0/3206831791 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fac64050940 con 0x7fac740d3310 2023-10-16T08:39:34.385 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.379+0000 7fac7bae2640 1 -- 172.21.15.40:0/3206831791 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fac740b7a60 con 0x7fac740d3310 2023-10-16T08:39:34.386 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.379+0000 7fac6a7fc640 1 -- 172.21.15.40:0/3206831791 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v22) v1 ==== 93+0+10 (secure 0 0 0) 0x7fac640170a0 con 0x7fac740d3310 2023-10-16T08:39:34.387 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 1 2023-10-16T08:39:34.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.383+0000 7fac7bae2640 1 -- 172.21.15.40:0/3206831791 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fac580664a0 msgr2=0x7fac58068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:34.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.383+0000 7fac7bae2640 1 --2- 172.21.15.40:0/3206831791 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fac580664a0 0x7fac58068960 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fac700af000 tx=0x7fac70008040 comp rx=0 tx=0).stop 2023-10-16T08:39:34.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.383+0000 7fac7bae2640 1 -- 172.21.15.40:0/3206831791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac740d3310 msgr2=0x7fac7414f570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:34.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.383+0000 7fac7bae2640 1 --2- 172.21.15.40:0/3206831791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac740d3310 0x7fac7414f570 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fac6400c490 tx=0x7fac64001e80 comp rx=0 tx=0).stop 2023-10-16T08:39:34.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.383+0000 7fac7bae2640 1 -- 172.21.15.40:0/3206831791 shutdown_connections 2023-10-16T08:39:34.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.383+0000 7fac7bae2640 1 --2- 172.21.15.40:0/3206831791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac7415cdc0 0x7fac7415f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.383+0000 7fac7bae2640 1 --2- 172.21.15.40:0/3206831791 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fac580664a0 0x7fac58068960 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.383+0000 7fac7bae2640 1 --2- 172.21.15.40:0/3206831791 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fac740d3310 0x7fac7414f570 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.383+0000 7fac7bae2640 1 --2- 172.21.15.40:0/3206831791 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fac740b8590 0x7fac741542f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.383+0000 7fac7bae2640 1 -- 172.21.15.40:0/3206831791 >> 172.21.15.40:0/3206831791 conn(0x7fac7406a290 msgr2=0x7fac7415c040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:34.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.383+0000 7fac7bae2640 1 -- 172.21.15.40:0/3206831791 shutdown_connections 2023-10-16T08:39:34.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.383+0000 7fac7bae2640 1 -- 172.21.15.40:0/3206831791 wait complete. 2023-10-16T08:39:34.404 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-16T08:39:34.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.515+0000 7f24728df640 1 Processor -- start 2023-10-16T08:39:34.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.515+0000 7f24728df640 1 -- start start 2023-10-16T08:39:34.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.515+0000 7f24728df640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f246c06bc60 0x7f246c0c8ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:34.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.515+0000 7f24728df640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f246c15d970 0x7f246c06c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:34.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.515+0000 7f24728df640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f246c0cc460 con 0x7f246c1598f0 2023-10-16T08:39:34.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.515+0000 7f24728df640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f246c0c9420 con 0x7f246c15d970 2023-10-16T08:39:34.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.515+0000 7f24728df640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246c0cca20 con 0x7f246c06bc60 2023-10-16T08:39:34.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.515+0000 7f246b7fe640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f246c15d970 0x7f246c06c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:34.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.515+0000 7f246bfff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f246c06bc60 0x7f246c0c8ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:34.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.515+0000 7f2470e7c640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f246c1598f0 0x7f246c159d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.40:54936/0 (socket says 172.21.15.40:54936) 2023-10-16T08:39:34.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.515+0000 7f246b7fe640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f246c15d970 0x7f246c06c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.40:46006/0 (socket says 172.21.15.40:46006) 2023-10-16T08:39:34.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.515+0000 7f2470e7c640 1 -- 172.21.15.40:0/2090913499 learned_addr learned my addr 172.21.15.40:0/2090913499 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:34.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.515+0000 7f246b7fe640 1 -- 172.21.15.40:0/2090913499 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f246c06bc60 msgr2=0x7f246c0c8ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:34.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.515+0000 7f246b7fe640 1 --2- 172.21.15.40:0/2090913499 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f246c06bc60 0x7f246c0c8ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.515+0000 7f246b7fe640 1 -- 172.21.15.40:0/2090913499 >> v1:172.21.15.86:6789/0 conn(0x7f246c1598f0 legacy=0x7f246c159d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.515+0000 7f246b7fe640 1 -- 172.21.15.40:0/2090913499 --> v2:172.21.15.86:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f246c06c580 con 0x7f246c15d970 2023-10-16T08:39:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f246b7fe640 1 --2- 172.21.15.40:0/2090913499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f246c15d970 0x7f246c06c040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f24600097c0 tx=0x7f246001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3f6fb92a020cc882 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:34.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f246affd640 1 -- 172.21.15.40:0/2090913499 <== mon.1 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24600ab650 con 0x7f246c15d970 2023-10-16T08:39:34.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f246affd640 1 -- 172.21.15.40:0/2090913499 <== mon.1 v2:172.21.15.86:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24600ab7b0 con 0x7f246c15d970 2023-10-16T08:39:34.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f246affd640 1 -- 172.21.15.40:0/2090913499 <== mon.1 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24600abb40 con 0x7f246c15d970 2023-10-16T08:39:34.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f24728df640 1 -- 172.21.15.40:0/2090913499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f246c15d970 msgr2=0x7f246c06c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f24728df640 1 --2- 172.21.15.40:0/2090913499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f246c15d970 0x7f246c06c040 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f24600097c0 tx=0x7f246001c450 comp rx=0 tx=0).stop 2023-10-16T08:39:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f24728df640 1 -- 172.21.15.40:0/2090913499 shutdown_connections 2023-10-16T08:39:34.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f24728df640 1 --2- 172.21.15.40:0/2090913499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f246c15d970 0x7f246c06c040 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f24728df640 1 --2- 172.21.15.40:0/2090913499 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f246c06bc60 0x7f246c0c8ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f24728df640 1 -- 172.21.15.40:0/2090913499 >> 172.21.15.40:0/2090913499 conn(0x7f246c0ce890 msgr2=0x7f246c0ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f24728df640 1 -- 172.21.15.40:0/2090913499 shutdown_connections 2023-10-16T08:39:34.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f24728df640 1 -- 172.21.15.40:0/2090913499 wait complete. 2023-10-16T08:39:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f24728df640 1 Processor -- start 2023-10-16T08:39:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f24728df640 1 -- start start 2023-10-16T08:39:34.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f24728df640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f246c06bc60 0x7f246c0c65a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:34.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f24728df640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f246c1598f0 0x7f246c0c85f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:34.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f24728df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f246c15d970 0x7f246c0c4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:34.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f24728df640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246c0c7820 con 0x7f246c1598f0 2023-10-16T08:39:34.532 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f24728df640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f246c0c8b30 con 0x7f246c15d970 2023-10-16T08:39:34.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f24728df640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f246c0c6c10 con 0x7f246c06bc60 2023-10-16T08:39:34.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f246b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f246c1598f0 0x7f246c0c85f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:34.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.519+0000 7f246b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f246c1598f0 0x7f246c0c85f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:49908/0 (socket says 172.21.15.40:49908) 2023-10-16T08:39:34.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.523+0000 7f246b7fe640 1 -- 172.21.15.40:0/726351910 learned_addr learned my addr 172.21.15.40:0/726351910 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:34.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.523+0000 7f2470e7c640 1 --2- 172.21.15.40:0/726351910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f246c15d970 0x7f246c0c4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:34.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.523+0000 7f246bfff640 1 --2- 172.21.15.40:0/726351910 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f246c06bc60 0x7f246c0c65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.523+0000 7f246b7fe640 1 -- 172.21.15.40:0/726351910 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f246c06bc60 msgr2=0x7f246c0c65a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.523+0000 7f246b7fe640 1 --2- 172.21.15.40:0/726351910 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f246c06bc60 0x7f246c0c65a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.523+0000 7f246b7fe640 1 -- 172.21.15.40:0/726351910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f246c15d970 msgr2=0x7f246c0c4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:34.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.523+0000 7f246b7fe640 1 --2- 172.21.15.40:0/726351910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f246c15d970 0x7f246c0c4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.523+0000 7f246b7fe640 1 -- 172.21.15.40:0/726351910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2460009420 con 0x7f246c1598f0 2023-10-16T08:39:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.523+0000 7f246b7fe640 1 --2- 172.21.15.40:0/726351910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f246c1598f0 0x7f246c0c85f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f246001a080 tx=0x7f246001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.523+0000 7f2468ff9640 1 -- 172.21.15.40:0/726351910 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24600b5810 con 0x7f246c1598f0 2023-10-16T08:39:34.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.523+0000 7f2468ff9640 1 -- 172.21.15.40:0/726351910 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24600b5970 con 0x7f246c1598f0 2023-10-16T08:39:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.523+0000 7f24728df640 1 -- 172.21.15.40:0/726351910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f246c0c6ef0 con 0x7f246c1598f0 2023-10-16T08:39:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.523+0000 7f2468ff9640 1 -- 172.21.15.40:0/726351910 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24600b5d20 con 0x7f246c1598f0 2023-10-16T08:39:34.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.523+0000 7f24728df640 1 -- 172.21.15.40:0/726351910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f246c0c74b0 con 0x7f246c1598f0 2023-10-16T08:39:34.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.523+0000 7f24728df640 1 -- 172.21.15.40:0/726351910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f246c067a20 con 0x7f246c1598f0 2023-10-16T08:39:34.538 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.523+0000 7f2468ff9640 1 -- 172.21.15.40:0/726351910 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f24600bf410 con 0x7f246c1598f0 2023-10-16T08:39:34.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.523+0000 7f2468ff9640 1 --2- 172.21.15.40:0/726351910 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f24580664a0 0x7f2458068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:34.539 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.523+0000 7f2468ff9640 1 -- 172.21.15.40:0/726351910 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f246011ee30 con 0x7f246c1598f0 2023-10-16T08:39:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.523+0000 7f246bfff640 1 --2- 172.21.15.40:0/726351910 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f24580664a0 0x7f2458068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.523+0000 7f246bfff640 1 --2- 172.21.15.40:0/726351910 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f24580664a0 0x7f2458068960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f246c1f9ce0 tx=0x7f245c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:34.540 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.527+0000 7f2468ff9640 1 -- 172.21.15.40:0/726351910 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f24600ec2b0 con 0x7f246c1598f0 2023-10-16T08:39:34.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.683+0000 7f24728df640 1 -- 172.21.15.40:0/726351910 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f246c0c8d30 con 0x7f246c1598f0 2023-10-16T08:39:34.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.683+0000 7f2468ff9640 1 -- 172.21.15.40:0/726351910 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v22) v1 ==== 104+0+11 (secure 0 0 0) 0x7f24600ef8d0 con 0x7f246c1598f0 2023-10-16T08:39:34.691 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 32 2023-10-16T08:39:34.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.687+0000 7f24728df640 1 -- 172.21.15.40:0/726351910 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f24580664a0 msgr2=0x7f2458068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:34.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.687+0000 7f24728df640 1 --2- 172.21.15.40:0/726351910 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f24580664a0 0x7f2458068960 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f246c1f9ce0 tx=0x7f245c008040 comp rx=0 tx=0).stop 2023-10-16T08:39:34.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.687+0000 7f24728df640 1 -- 172.21.15.40:0/726351910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f246c1598f0 msgr2=0x7f246c0c85f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:34.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.687+0000 7f24728df640 1 --2- 172.21.15.40:0/726351910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f246c1598f0 0x7f246c0c85f0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f246001a080 tx=0x7f246001ac90 comp rx=0 tx=0).stop 2023-10-16T08:39:34.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.687+0000 7f24728df640 1 -- 172.21.15.40:0/726351910 shutdown_connections 2023-10-16T08:39:34.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.687+0000 7f24728df640 1 --2- 172.21.15.40:0/726351910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f246c15d970 0x7f246c0c4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.687+0000 7f24728df640 1 --2- 172.21.15.40:0/726351910 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f24580664a0 0x7f2458068960 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.687+0000 7f24728df640 1 --2- 172.21.15.40:0/726351910 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f246c1598f0 0x7f246c0c85f0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.687+0000 7f24728df640 1 --2- 172.21.15.40:0/726351910 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f246c06bc60 0x7f246c0c65a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.687+0000 7f24728df640 1 -- 172.21.15.40:0/726351910 >> 172.21.15.40:0/726351910 conn(0x7f246c0ce890 msgr2=0x7f246c0ada10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:34.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.687+0000 7f24728df640 1 -- 172.21.15.40:0/726351910 shutdown_connections 2023-10-16T08:39:34.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.687+0000 7f24728df640 1 -- 172.21.15.40:0/726351910 wait complete. 2023-10-16T08:39:34.709 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-16T08:39:34.839 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.835+0000 7fb00f758640 1 Processor -- start 2023-10-16T08:39:34.840 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.835+0000 7fb00f758640 1 -- start start 2023-10-16T08:39:34.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.835+0000 7fb00f758640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb00815cea0 0x7fb0081609c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:34.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.835+0000 7fb00f758640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb0080b8550 0x7fb0080cd920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.835+0000 7fb00f758640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0080d11c0 con 0x7fb0080cde60 2023-10-16T08:39:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.835+0000 7fb00f758640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0080ce2d0 con 0x7fb00815cea0 2023-10-16T08:39:34.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.835+0000 7fb00f758640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0081618f0 con 0x7fb0080b8550 2023-10-16T08:39:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.835+0000 7fb00d4f4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb00815cea0 0x7fb0081609c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.835+0000 7fb00d4f4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fb00815cea0 0x7fb0081609c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50648/0 (socket says 172.21.15.40:50648) 2023-10-16T08:39:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.835+0000 7fb00d4f4640 1 -- 172.21.15.40:0/3057380475 learned_addr learned my addr 172.21.15.40:0/3057380475 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:34.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.835+0000 7fb00dcf5640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb0080cde60 0x7fb0080b45d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45604/0 (socket says 172.21.15.40:45604) 2023-10-16T08:39:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.835+0000 7fb00ccf3640 1 --2- 172.21.15.40:0/3057380475 >> v2:172.21.15.86:3300/0 conn(0x7fb0080b8550 0x7fb0080cd920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.835+0000 7fb00d4f4640 1 -- 172.21.15.40:0/3057380475 >> v2:172.21.15.86:3300/0 conn(0x7fb0080b8550 msgr2=0x7fb0080cd920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:34.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.835+0000 7fb00d4f4640 1 --2- 172.21.15.40:0/3057380475 >> v2:172.21.15.86:3300/0 conn(0x7fb0080b8550 0x7fb0080cd920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.835+0000 7fb00d4f4640 1 -- 172.21.15.40:0/3057380475 >> v1:172.21.15.40:6789/0 conn(0x7fb0080cde60 legacy=0x7fb0080b45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:34.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.835+0000 7fb00d4f4640 1 -- 172.21.15.40:0/3057380475 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb00815d280 con 0x7fb00815cea0 2023-10-16T08:39:34.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.835+0000 7faffffff640 1 -- 172.21.15.40:0/3057380475 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1544936710 0 0) 0x7fb0080d11c0 con 0x7fb0080cde60 2023-10-16T08:39:34.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00d4f4640 1 --2- 172.21.15.40:0/3057380475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb00815cea0 0x7fb0081609c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb0080b8cc0 tx=0x7fb00401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5703a2e205d2a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:34.848 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7faffffff640 1 -- 172.21.15.40:0/3057380475 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0040ab5f0 con 0x7fb00815cea0 2023-10-16T08:39:34.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7faffffff640 1 -- 172.21.15.40:0/3057380475 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0040ab750 con 0x7fb00815cea0 2023-10-16T08:39:34.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7faffffff640 1 -- 172.21.15.40:0/3057380475 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0040abae0 con 0x7fb00815cea0 2023-10-16T08:39:34.849 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00f758640 1 -- 172.21.15.40:0/3057380475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb00815cea0 msgr2=0x7fb0081609c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:34.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00f758640 1 --2- 172.21.15.40:0/3057380475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb00815cea0 0x7fb0081609c0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fb0080b8cc0 tx=0x7fb00401c4d0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00f758640 1 -- 172.21.15.40:0/3057380475 shutdown_connections 2023-10-16T08:39:34.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00f758640 1 --2- 172.21.15.40:0/3057380475 >> v2:172.21.15.86:3300/0 conn(0x7fb0080b8550 0x7fb0080cd920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00f758640 1 --2- 172.21.15.40:0/3057380475 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb00815cea0 0x7fb0081609c0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00f758640 1 -- 172.21.15.40:0/3057380475 >> 172.21.15.40:0/3057380475 conn(0x7fb008065a80 msgr2=0x7fb008065e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00f758640 1 -- 172.21.15.40:0/3057380475 shutdown_connections 2023-10-16T08:39:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00f758640 1 -- 172.21.15.40:0/3057380475 wait complete. 2023-10-16T08:39:34.851 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00f758640 1 Processor -- start 2023-10-16T08:39:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00f758640 1 -- start start 2023-10-16T08:39:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00f758640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0080b8550 0x7fb0080c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00f758640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0080cde60 0x7fb008153870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:34.852 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00f758640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb00815cea0 0x7fb008153db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:34.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00f758640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080b7c30 con 0x7fb0080cde60 2023-10-16T08:39:34.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00f758640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080b70b0 con 0x7fb0080b8550 2023-10-16T08:39:34.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00f758640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0080b7390 con 0x7fb00815cea0 2023-10-16T08:39:34.853 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00ccf3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0080cde60 0x7fb008153870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:34.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00ccf3640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0080cde60 0x7fb008153870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50658/0 (socket says 172.21.15.40:50658) 2023-10-16T08:39:34.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00ccf3640 1 -- 172.21.15.40:0/4000241161 learned_addr learned my addr 172.21.15.40:0/4000241161 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:34.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00dcf5640 1 --2- 172.21.15.40:0/4000241161 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb00815cea0 0x7fb008153db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:34.854 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00d4f4640 1 --2- 172.21.15.40:0/4000241161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0080b8550 0x7fb0080c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00ccf3640 1 -- 172.21.15.40:0/4000241161 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb00815cea0 msgr2=0x7fb008153db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00ccf3640 1 --2- 172.21.15.40:0/4000241161 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb00815cea0 0x7fb008153db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00ccf3640 1 -- 172.21.15.40:0/4000241161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0080b8550 msgr2=0x7fb0080c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:34.855 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00ccf3640 1 --2- 172.21.15.40:0/4000241161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0080b8550 0x7fb0080c44c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:34.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.839+0000 7fb00ccf3640 1 -- 172.21.15.40:0/4000241161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0040094d0 con 0x7fb0080cde60 2023-10-16T08:39:34.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.843+0000 7fb00ccf3640 1 --2- 172.21.15.40:0/4000241161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0080cde60 0x7fb008153870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7faff800d7d0 tx=0x7faff800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:34.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.843+0000 7faffdffb640 1 -- 172.21.15.40:0/4000241161 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff800b6c0 con 0x7fb0080cde60 2023-10-16T08:39:34.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.843+0000 7faffdffb640 1 -- 172.21.15.40:0/4000241161 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff800b820 con 0x7fb0080cde60 2023-10-16T08:39:34.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.843+0000 7fb00f758640 1 -- 172.21.15.40:0/4000241161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0080b7670 con 0x7fb0080cde60 2023-10-16T08:39:34.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.843+0000 7faffdffb640 1 -- 172.21.15.40:0/4000241161 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff800bbb0 con 0x7fb0080cde60 2023-10-16T08:39:34.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.843+0000 7fb00f758640 1 -- 172.21.15.40:0/4000241161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0080b6cd0 con 0x7fb0080cde60 2023-10-16T08:39:34.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.843+0000 7fb00f758640 1 -- 172.21.15.40:0/4000241161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb00804e9c0 con 0x7fb0080cde60 2023-10-16T08:39:34.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.843+0000 7faffdffb640 1 -- 172.21.15.40:0/4000241161 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7faff800bd10 con 0x7fb0080cde60 2023-10-16T08:39:34.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.843+0000 7faffdffb640 1 --2- 172.21.15.40:0/4000241161 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fafec0664d0 0x7fafec068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:34.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.843+0000 7faffdffb640 1 -- 172.21.15.40:0/4000241161 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7faff80843c0 con 0x7fb0080cde60 2023-10-16T08:39:34.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.843+0000 7fb00d4f4640 1 --2- 172.21.15.40:0/4000241161 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fafec0664d0 0x7fafec068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:34.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.843+0000 7fb00d4f4640 1 --2- 172.21.15.40:0/4000241161 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fafec0664d0 0x7fafec068990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb0040af000 tx=0x7fb004008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:34.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:34.847+0000 7faffdffb640 1 -- 172.21.15.40:0/4000241161 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7faff80517b0 con 0x7fb0080cde60 2023-10-16T08:39:35.008 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.003+0000 7fb00f758640 1 -- 172.21.15.40:0/4000241161 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb0080d08d0 con 0x7fb0080cde60 2023-10-16T08:39:35.009 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.003+0000 7faffdffb640 1 -- 172.21.15.40:0/4000241161 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v22) v1 ==== 100+0+11 (secure 0 0 0) 0x7faff8054dd0 con 0x7fb0080cde60 2023-10-16T08:39:35.009 INFO:teuthology.orchestra.run.smithi040.stdout:pg_num: 64 2023-10-16T08:39:35.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.003+0000 7fb00f758640 1 -- 172.21.15.40:0/4000241161 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fafec0664d0 msgr2=0x7fafec068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:35.011 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.007+0000 7fb00f758640 1 --2- 172.21.15.40:0/4000241161 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fafec0664d0 0x7fafec068990 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fb0040af000 tx=0x7fb004008040 comp rx=0 tx=0).stop 2023-10-16T08:39:35.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.007+0000 7fb00f758640 1 -- 172.21.15.40:0/4000241161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0080cde60 msgr2=0x7fb008153870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:35.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.007+0000 7fb00f758640 1 --2- 172.21.15.40:0/4000241161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0080cde60 0x7fb008153870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7faff800d7d0 tx=0x7faff800dc90 comp rx=0 tx=0).stop 2023-10-16T08:39:35.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.007+0000 7fb00f758640 1 -- 172.21.15.40:0/4000241161 shutdown_connections 2023-10-16T08:39:35.012 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.007+0000 7fb00f758640 1 --2- 172.21.15.40:0/4000241161 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb00815cea0 0x7fb008153db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:35.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.007+0000 7fb00f758640 1 --2- 172.21.15.40:0/4000241161 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fafec0664d0 0x7fafec068990 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:35.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.007+0000 7fb00f758640 1 --2- 172.21.15.40:0/4000241161 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb0080cde60 0x7fb008153870 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:35.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.007+0000 7fb00f758640 1 --2- 172.21.15.40:0/4000241161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0080b8550 0x7fb0080c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:35.013 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.007+0000 7fb00f758640 1 -- 172.21.15.40:0/4000241161 >> 172.21.15.40:0/4000241161 conn(0x7fb008065a80 msgr2=0x7fb0080d1e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:35.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.007+0000 7fb00f758640 1 -- 172.21.15.40:0/4000241161 shutdown_connections 2023-10-16T08:39:35.014 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.007+0000 7fb00f758640 1 -- 172.21.15.40:0/4000241161 wait complete. 2023-10-16T08:39:35.027 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-10-16T08:39:35.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.143+0000 7fda44398640 1 Processor -- start 2023-10-16T08:39:35.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.143+0000 7fda44398640 1 -- start start 2023-10-16T08:39:35.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda44398640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fda3c15d180 0x7fda3c06b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:35.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda44398640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda3c0cf480 con 0x7fda3c0ad820 2023-10-16T08:39:35.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda44398640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda3c0cf760 con 0x7fda3c15d560 2023-10-16T08:39:35.154 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda44398640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda3c0c8fb0 con 0x7fda3c15d180 2023-10-16T08:39:35.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda42935640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fda3c0ad820 0x7fda3c0cd220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45606/0 (socket says 172.21.15.40:45606) 2023-10-16T08:39:35.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda42935640 1 -- 172.21.15.40:0/3773702309 learned_addr learned my addr 172.21.15.40:0/3773702309 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:35.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda42134640 1 --2- 172.21.15.40:0/3773702309 >> v2:172.21.15.86:3300/0 conn(0x7fda3c15d180 0x7fda3c06b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:35.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41132640 1 -- 172.21.15.40:0/3773702309 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4170010575 0 0) 0x7fda3c0cf480 con 0x7fda3c0ad820 2023-10-16T08:39:35.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41132640 1 -- 172.21.15.40:0/3773702309 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda24003530 con 0x7fda3c0ad820 2023-10-16T08:39:35.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41132640 1 -- 172.21.15.40:0/3773702309 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3976801463 0 0) 0x7fda24003530 con 0x7fda3c0ad820 2023-10-16T08:39:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41132640 1 -- 172.21.15.40:0/3773702309 >> v2:172.21.15.86:3300/0 conn(0x7fda3c15d180 msgr2=0x7fda3c06b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41132640 1 --2- 172.21.15.40:0/3773702309 >> v2:172.21.15.86:3300/0 conn(0x7fda3c15d180 0x7fda3c06b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41132640 1 -- 172.21.15.40:0/3773702309 >> v1:172.21.15.86:6789/0 conn(0x7fda3c15d560 legacy=0x7fda3c0c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:35.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41132640 1 -- 172.21.15.40:0/3773702309 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c0c9290 con 0x7fda3c0ad820 2023-10-16T08:39:35.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41132640 1 -- 172.21.15.40:0/3773702309 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7fda34001680 con 0x7fda3c0ad820 2023-10-16T08:39:35.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41132640 1 -- 172.21.15.40:0/3773702309 >> v1:172.21.15.40:6789/0 conn(0x7fda3c0ad820 legacy=0x7fda3c0cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:35.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41132640 1 --2- 172.21.15.40:0/3773702309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda240039f0 0x7fda24003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:35.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41132640 1 --2- 172.21.15.40:0/3773702309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda240047d0 0x7fda24004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:35.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41132640 1 --2- 172.21.15.40:0/3773702309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda24005190 0x7fda2400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:35.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41132640 1 -- 172.21.15.40:0/3773702309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c0c8fb0 con 0x7fda240047d0 2023-10-16T08:39:35.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41132640 1 -- 172.21.15.40:0/3773702309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda2400c0c0 con 0x7fda24005190 2023-10-16T08:39:35.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41132640 1 -- 172.21.15.40:0/3773702309 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda2400c280 con 0x7fda240039f0 2023-10-16T08:39:35.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41933640 1 --2- 172.21.15.40:0/3773702309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda240047d0 0x7fda24004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:35.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda42134640 1 --2- 172.21.15.40:0/3773702309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda240039f0 0x7fda24003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:35.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41933640 1 -- 172.21.15.40:0/3773702309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda240039f0 msgr2=0x7fda24003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:35.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41933640 1 --2- 172.21.15.40:0/3773702309 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda240039f0 0x7fda24003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:35.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda42935640 1 --2- 172.21.15.40:0/3773702309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda24005190 0x7fda2400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:35.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41933640 1 -- 172.21.15.40:0/3773702309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda24005190 msgr2=0x7fda2400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:35.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41933640 1 --2- 172.21.15.40:0/3773702309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda24005190 0x7fda2400b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:35.161 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda42935640 1 -- 172.21.15.40:0/3773702309 reap_dead start 2023-10-16T08:39:35.162 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.147+0000 7fda41933640 1 -- 172.21.15.40:0/3773702309 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda24003530 con 0x7fda240047d0 2023-10-16T08:39:35.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda41933640 1 --2- 172.21.15.40:0/3773702309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda240047d0 0x7fda24004c50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fda2c001780 tx=0x7fda2c01c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b17a7d867ba6df9e server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:35.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda41132640 1 -- 172.21.15.40:0/3773702309 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c0052d0 con 0x7fda240047d0 2023-10-16T08:39:35.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda41132640 1 -- 172.21.15.40:0/3773702309 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda2c0a7050 con 0x7fda240047d0 2023-10-16T08:39:35.166 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda41132640 1 -- 172.21.15.40:0/3773702309 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda2c003600 con 0x7fda240047d0 2023-10-16T08:39:35.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda44398640 1 -- 172.21.15.40:0/3773702309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda240047d0 msgr2=0x7fda24004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:35.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda44398640 1 --2- 172.21.15.40:0/3773702309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda240047d0 0x7fda24004c50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fda2c001780 tx=0x7fda2c01c3e0 comp rx=0 tx=0).stop 2023-10-16T08:39:35.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda44398640 1 -- 172.21.15.40:0/3773702309 shutdown_connections 2023-10-16T08:39:35.167 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda44398640 1 --2- 172.21.15.40:0/3773702309 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda240047d0 0x7fda24004c50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:35.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda44398640 1 -- 172.21.15.40:0/3773702309 >> 172.21.15.40:0/3773702309 conn(0x7fda3c0ce960 msgr2=0x7fda3c0ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:35.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda44398640 1 -- 172.21.15.40:0/3773702309 shutdown_connections 2023-10-16T08:39:35.168 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda44398640 1 -- 172.21.15.40:0/3773702309 wait complete. 2023-10-16T08:39:35.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda44398640 1 Processor -- start 2023-10-16T08:39:35.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda44398640 1 -- start start 2023-10-16T08:39:35.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda44398640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda240047d0 0x7fda3c15b450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:35.169 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda44398640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda3c159010 0x7fda3c1f08e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:35.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda44398640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3c1f0e20 0x7fda3c1f12a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:35.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda44398640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c0c6fe0 con 0x7fda240047d0 2023-10-16T08:39:35.170 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda44398640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c15c170 con 0x7fda3c1f0e20 2023-10-16T08:39:35.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda44398640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c15b990 con 0x7fda3c159010 2023-10-16T08:39:35.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda42134640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda240047d0 0x7fda3c15b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:35.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda42935640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3c1f0e20 0x7fda3c1f12a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:35.171 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda42134640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda240047d0 0x7fda3c15b450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50666/0 (socket says 172.21.15.40:50666) 2023-10-16T08:39:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda41933640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda3c159010 0x7fda3c1f08e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda42134640 1 -- 172.21.15.40:0/4279552603 learned_addr learned my addr 172.21.15.40:0/4279552603 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda42134640 1 -- 172.21.15.40:0/4279552603 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda3c159010 msgr2=0x7fda3c1f08e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:35.172 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda42134640 1 --2- 172.21.15.40:0/4279552603 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda3c159010 0x7fda3c1f08e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda42134640 1 -- 172.21.15.40:0/4279552603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3c1f0e20 msgr2=0x7fda3c1f12a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda42134640 1 --2- 172.21.15.40:0/4279552603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3c1f0e20 0x7fda3c1f12a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:35.173 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda42134640 1 -- 172.21.15.40:0/4279552603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c15bbb0 con 0x7fda240047d0 2023-10-16T08:39:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.151+0000 7fda42935640 1 --2- 172.21.15.40:0/4279552603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3c1f0e20 0x7fda3c1f12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.155+0000 7fda42134640 1 --2- 172.21.15.40:0/4279552603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda240047d0 0x7fda3c15b450 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fda30007f30 tx=0x7fda30011660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.155+0000 7fda3affd640 1 -- 172.21.15.40:0/4279552603 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda3000e030 con 0x7fda240047d0 2023-10-16T08:39:35.174 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.155+0000 7fda3affd640 1 -- 172.21.15.40:0/4279552603 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda30016470 con 0x7fda240047d0 2023-10-16T08:39:35.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.155+0000 7fda44398640 1 -- 172.21.15.40:0/4279552603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda3c15be90 con 0x7fda240047d0 2023-10-16T08:39:35.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.155+0000 7fda3affd640 1 -- 172.21.15.40:0/4279552603 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda30016820 con 0x7fda240047d0 2023-10-16T08:39:35.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.155+0000 7fda44398640 1 -- 172.21.15.40:0/4279552603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda3c0c2760 con 0x7fda240047d0 2023-10-16T08:39:35.175 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.155+0000 7fda44398640 1 -- 172.21.15.40:0/4279552603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda3c04e9c0 con 0x7fda240047d0 2023-10-16T08:39:35.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.155+0000 7fda3affd640 1 -- 172.21.15.40:0/4279552603 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fda30016980 con 0x7fda240047d0 2023-10-16T08:39:35.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.155+0000 7fda3affd640 1 --2- 172.21.15.40:0/4279552603 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fda280664a0 0x7fda28068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:35.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.155+0000 7fda3affd640 1 -- 172.21.15.40:0/4279552603 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fda3000f070 con 0x7fda240047d0 2023-10-16T08:39:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.155+0000 7fda41933640 1 --2- 172.21.15.40:0/4279552603 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fda280664a0 0x7fda28068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.155+0000 7fda41933640 1 --2- 172.21.15.40:0/4279552603 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fda280664a0 0x7fda28068960 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fda2c001750 tx=0x7fda2c0011a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:35.177 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.159+0000 7fda3affd640 1 -- 172.21.15.40:0/4279552603 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fda30052230 con 0x7fda240047d0 2023-10-16T08:39:35.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.331+0000 7fda44398640 1 -- 172.21.15.40:0/4279552603 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fda3c0ade40 con 0x7fda240047d0 2023-10-16T08:39:35.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.331+0000 7fda3affd640 1 -- 172.21.15.40:0/4279552603 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v22) v1 ==== 80+0+0 (secure 0 0 0) 0x7fda30055850 con 0x7fda240047d0 2023-10-16T08:39:35.338 INFO:teuthology.orchestra.run.smithi040.stderr:listed 0 entries 2023-10-16T08:39:35.340 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.335+0000 7fda44398640 1 -- 172.21.15.40:0/4279552603 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fda280664a0 msgr2=0x7fda28068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:35.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.335+0000 7fda44398640 1 --2- 172.21.15.40:0/4279552603 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fda280664a0 0x7fda28068960 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fda2c001750 tx=0x7fda2c0011a0 comp rx=0 tx=0).stop 2023-10-16T08:39:35.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.335+0000 7fda44398640 1 -- 172.21.15.40:0/4279552603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda240047d0 msgr2=0x7fda3c15b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:35.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.335+0000 7fda44398640 1 --2- 172.21.15.40:0/4279552603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda240047d0 0x7fda3c15b450 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fda30007f30 tx=0x7fda30011660 comp rx=0 tx=0).stop 2023-10-16T08:39:35.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.335+0000 7fda44398640 1 -- 172.21.15.40:0/4279552603 shutdown_connections 2023-10-16T08:39:35.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.335+0000 7fda44398640 1 --2- 172.21.15.40:0/4279552603 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3c1f0e20 0x7fda3c1f12a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:35.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.335+0000 7fda44398640 1 --2- 172.21.15.40:0/4279552603 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda3c159010 0x7fda3c1f08e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:35.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.335+0000 7fda44398640 1 --2- 172.21.15.40:0/4279552603 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fda280664a0 0x7fda28068960 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:35.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.335+0000 7fda44398640 1 --2- 172.21.15.40:0/4279552603 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda240047d0 0x7fda3c15b450 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:35.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.335+0000 7fda44398640 1 -- 172.21.15.40:0/4279552603 >> 172.21.15.40:0/4279552603 conn(0x7fda3c0ce960 msgr2=0x7fda3c0cc510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:35.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.335+0000 7fda44398640 1 -- 172.21.15.40:0/4279552603 shutdown_connections 2023-10-16T08:39:35.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:35.335+0000 7fda44398640 1 -- 172.21.15.40:0/4279552603 wait complete. 2023-10-16T08:39:35.357 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2023-10-16T08:39:35.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:35.357 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-10-16T08:39:35.371 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi086.stderr:ceph-fuse[44175]: fuse finished with error 0 and tester_r 0 2023-10-16T08:39:35.372 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:39:41.475 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2023-10-16T08:39:41.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:41.476 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-16T08:39:41.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:41.484 DEBUG:teuthology.orchestra.run.smithi040:> ip netns list 2023-10-16T08:39:41.491 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-16T08:39:41.492 INFO:teuthology.orchestra.run.smithi040.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-16T08:39:41.492 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:41.492 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-16T08:39:41.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:41.550 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-16T08:39:41.607 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:41.608 DEBUG:teuthology.orchestra.run.smithi040:> sudo ip link delete ceph-brx 2023-10-16T08:39:41.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:41.770 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2023-10-16T08:39:41.777 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-16T08:39:41.777 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-16T08:39:41.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:41.778 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-16T08:39:41.833 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:41.833 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-16T08:39:41.920 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-16T08:39:41.920 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2023-10-16T08:39:42.129 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-16T08:39:42.139 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-16T08:39:42.139 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:39:42.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ec667d640 1 Processor -- start 2023-10-16T08:39:42.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ec667d640 1 -- start start 2023-10-16T08:39:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ec667d640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f1ec015b040 0x7f1ec00ca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:42.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ec667d640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec00adca0 con 0x7f1ec00adec0 2023-10-16T08:39:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ec667d640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ec00c5840 con 0x7f1ec006c7c0 2023-10-16T08:39:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ec667d640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ec00c9dd0 con 0x7f1ec015b040 2023-10-16T08:39:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebf7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1ec00adec0 0x7f1ec00c8fb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45608/0 (socket says 172.21.15.40:45608) 2023-10-16T08:39:42.279 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebf7fe640 1 -- 172.21.15.40:0/2338960573 learned_addr learned my addr 172.21.15.40:0/2338960573 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:42.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebffff640 1 --2- 172.21.15.40:0/2338960573 >> v2:172.21.15.86:3300/0 conn(0x7f1ec015b040 0x7f1ec00ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:42.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebeffd640 1 -- 172.21.15.40:0/2338960573 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 832632913 0 0) 0x7f1ec00adca0 con 0x7f1ec00adec0 2023-10-16T08:39:42.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebeffd640 1 -- 172.21.15.40:0/2338960573 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1ea8003530 con 0x7f1ec00adec0 2023-10-16T08:39:42.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebeffd640 1 -- 172.21.15.40:0/2338960573 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 133036157 0 0) 0x7f1ea8003530 con 0x7f1ec00adec0 2023-10-16T08:39:42.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebeffd640 1 -- 172.21.15.40:0/2338960573 >> v2:172.21.15.86:3300/0 conn(0x7f1ec015b040 msgr2=0x7f1ec00ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:42.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebeffd640 1 --2- 172.21.15.40:0/2338960573 >> v2:172.21.15.86:3300/0 conn(0x7f1ec015b040 0x7f1ec00ca740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:42.281 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebeffd640 1 -- 172.21.15.40:0/2338960573 >> v1:172.21.15.86:6789/0 conn(0x7f1ec006c7c0 legacy=0x7f1ec00c96c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:42.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebeffd640 1 -- 172.21.15.40:0/2338960573 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec00cf890 con 0x7f1ec00adec0 2023-10-16T08:39:42.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebeffd640 1 -- 172.21.15.40:0/2338960573 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f1eb0001680 con 0x7f1ec00adec0 2023-10-16T08:39:42.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebeffd640 1 -- 172.21.15.40:0/2338960573 >> v1:172.21.15.40:6789/0 conn(0x7f1ec00adec0 legacy=0x7f1ec00c8fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:42.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebeffd640 1 --2- 172.21.15.40:0/2338960573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ea8003a40 0x7f1ea8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:42.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebeffd640 1 --2- 172.21.15.40:0/2338960573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ea80047d0 0x7f1ea8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:42.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebeffd640 1 --2- 172.21.15.40:0/2338960573 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ea80052e0 0x7f1ea800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:42.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebeffd640 1 -- 172.21.15.40:0/2338960573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec00c9dd0 con 0x7f1ea80047d0 2023-10-16T08:39:42.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebeffd640 1 -- 172.21.15.40:0/2338960573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea800c220 con 0x7f1ea8003a40 2023-10-16T08:39:42.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebeffd640 1 -- 172.21.15.40:0/2338960573 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea800c3e0 con 0x7f1ea80052e0 2023-10-16T08:39:42.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebf7fe640 1 --2- 172.21.15.40:0/2338960573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ea80047d0 0x7f1ea8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:42.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebffff640 1 --2- 172.21.15.40:0/2338960573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ea8003a40 0x7f1ea8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:42.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ec4c1a640 1 --2- 172.21.15.40:0/2338960573 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ea80052e0 0x7f1ea800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:42.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebf7fe640 1 -- 172.21.15.40:0/2338960573 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ea80052e0 msgr2=0x7f1ea800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:42.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebf7fe640 1 --2- 172.21.15.40:0/2338960573 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ea80052e0 0x7f1ea800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:42.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebf7fe640 1 -- 172.21.15.40:0/2338960573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ea8003a40 msgr2=0x7f1ea8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:42.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebf7fe640 1 --2- 172.21.15.40:0/2338960573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ea8003a40 0x7f1ea8003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:42.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ebf7fe640 1 -- 172.21.15.40:0/2338960573 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea8003530 con 0x7f1ea80047d0 2023-10-16T08:39:42.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ec4c1a640 1 --2- 172.21.15.40:0/2338960573 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ea80052e0 0x7f1ea800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:42.287 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.271+0000 7f1ec4c1a640 1 -- 172.21.15.40:0/2338960573 reap_dead start 2023-10-16T08:39:42.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.275+0000 7f1ebf7fe640 1 --2- 172.21.15.40:0/2338960573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ea80047d0 0x7f1ea8004c50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000c0e0 tx=0x7f1eb001cb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c9ddd18cdc77c4c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:42.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.275+0000 7f1ebeffd640 1 -- 172.21.15.40:0/2338960573 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb0005ae0 con 0x7f1ea80047d0 2023-10-16T08:39:42.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.275+0000 7f1ebeffd640 1 -- 172.21.15.40:0/2338960573 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb0005c40 con 0x7f1ea80047d0 2023-10-16T08:39:42.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.275+0000 7f1ebeffd640 1 -- 172.21.15.40:0/2338960573 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb00072b0 con 0x7f1ea80047d0 2023-10-16T08:39:42.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.275+0000 7f1ec667d640 1 -- 172.21.15.40:0/2338960573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ea80047d0 msgr2=0x7f1ea8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:42.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.275+0000 7f1ec667d640 1 --2- 172.21.15.40:0/2338960573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ea80047d0 0x7f1ea8004c50 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f1eb000c0e0 tx=0x7f1eb001cb30 comp rx=0 tx=0).stop 2023-10-16T08:39:42.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.275+0000 7f1ec667d640 1 -- 172.21.15.40:0/2338960573 shutdown_connections 2023-10-16T08:39:42.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.275+0000 7f1ec667d640 1 --2- 172.21.15.40:0/2338960573 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ea80047d0 0x7f1ea8004c50 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:42.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.275+0000 7f1ec667d640 1 -- 172.21.15.40:0/2338960573 >> 172.21.15.40:0/2338960573 conn(0x7f1ec00ce960 msgr2=0x7f1ec00ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:42.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.275+0000 7f1ec667d640 1 -- 172.21.15.40:0/2338960573 shutdown_connections 2023-10-16T08:39:42.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.275+0000 7f1ec667d640 1 -- 172.21.15.40:0/2338960573 wait complete. 2023-10-16T08:39:42.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.275+0000 7f1ec667d640 1 Processor -- start 2023-10-16T08:39:42.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.275+0000 7f1ec667d640 1 -- start start 2023-10-16T08:39:42.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.275+0000 7f1ec667d640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ea80047d0 0x7f1ec014fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:42.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.275+0000 7f1ec667d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec01529a0 0x7f1ec00c36f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:42.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.275+0000 7f1ec667d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ec01594a0 0x7f1ec0159930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:42.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.275+0000 7f1ec667d640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec00c6f80 con 0x7f1ec01529a0 2023-10-16T08:39:42.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.275+0000 7f1ec667d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec0159e70 con 0x7f1ec01594a0 2023-10-16T08:39:42.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.275+0000 7f1ec667d640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ec00c66e0 con 0x7f1ea80047d0 2023-10-16T08:39:42.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ebf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec01529a0 0x7f1ec00c36f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:42.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ebf7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec01529a0 0x7f1ec00c36f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50696/0 (socket says 172.21.15.40:50696) 2023-10-16T08:39:42.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ebf7fe640 1 -- 172.21.15.40:0/2258048981 learned_addr learned my addr 172.21.15.40:0/2258048981 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:42.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ebffff640 1 --2- 172.21.15.40:0/2258048981 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ea80047d0 0x7f1ec014fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:42.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ec4c1a640 1 --2- 172.21.15.40:0/2258048981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ec01594a0 0x7f1ec0159930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:42.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ebf7fe640 1 -- 172.21.15.40:0/2258048981 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ea80047d0 msgr2=0x7f1ec014fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:42.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ebf7fe640 1 --2- 172.21.15.40:0/2258048981 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ea80047d0 0x7f1ec014fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:42.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ebf7fe640 1 -- 172.21.15.40:0/2258048981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ec01594a0 msgr2=0x7f1ec0159930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:42.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ebf7fe640 1 --2- 172.21.15.40:0/2258048981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ec01594a0 0x7f1ec0159930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:42.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ebf7fe640 1 -- 172.21.15.40:0/2258048981 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ec00c69c0 con 0x7f1ec01529a0 2023-10-16T08:39:42.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ebf7fe640 1 --2- 172.21.15.40:0/2258048981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec01529a0 0x7f1ec00c36f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1eb001c960 tx=0x7f1ea8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:42.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ebcff9640 1 -- 172.21.15.40:0/2258048981 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb00b4070 con 0x7f1ec01529a0 2023-10-16T08:39:42.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ebcff9640 1 -- 172.21.15.40:0/2258048981 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1eb0001480 con 0x7f1ec01529a0 2023-10-16T08:39:42.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ec667d640 1 -- 172.21.15.40:0/2258048981 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ec00c6ca0 con 0x7f1ec01529a0 2023-10-16T08:39:42.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ebcff9640 1 -- 172.21.15.40:0/2258048981 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1eb00072b0 con 0x7f1ec01529a0 2023-10-16T08:39:42.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ec667d640 1 -- 172.21.15.40:0/2258048981 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ec01503d0 con 0x7f1ec01529a0 2023-10-16T08:39:42.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ec667d640 1 -- 172.21.15.40:0/2258048981 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ec004e9c0 con 0x7f1ec01529a0 2023-10-16T08:39:42.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ebcff9640 1 -- 172.21.15.40:0/2258048981 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1eb0007410 con 0x7f1ec01529a0 2023-10-16T08:39:42.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ebcff9640 1 --2- 172.21.15.40:0/2258048981 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1eac0664d0 0x7f1eac068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:42.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ebcff9640 1 -- 172.21.15.40:0/2258048981 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f1eb011d250 con 0x7f1ec01529a0 2023-10-16T08:39:42.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ebffff640 1 --2- 172.21.15.40:0/2258048981 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1eac0664d0 0x7f1eac068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:42.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.279+0000 7f1ebffff640 1 --2- 172.21.15.40:0/2258048981 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1eac0664d0 0x7f1eac068990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1ec015a0c0 tx=0x7f1eb4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:42.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.283+0000 7f1ebcff9640 1 -- 172.21.15.40:0/2258048981 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f1eb00ea640 con 0x7f1ec01529a0 2023-10-16T08:39:42.443 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.439+0000 7f1ec667d640 1 -- 172.21.15.40:0/2258048981 --> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1ec00ad970 con 0x7f1eac0664d0 2023-10-16T08:39:42.466 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.459+0000 7f1ebcff9640 1 -- 172.21.15.40:0/2258048981 <== mgr.4111 v2:172.21.15.86:6832/1188970899 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286974 (secure 0 0 0) 0x7f1eb4009810 con 0x7f1eac0664d0 2023-10-16T08:39:42.469 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:39:42.471 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-16T08:39:42.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.463+0000 7f1ec667d640 1 -- 172.21.15.40:0/2258048981 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1eac0664d0 msgr2=0x7f1eac068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:42.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.463+0000 7f1ec667d640 1 --2- 172.21.15.40:0/2258048981 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1eac0664d0 0x7f1eac068990 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f1ec015a0c0 tx=0x7f1eb4008040 comp rx=0 tx=0).stop 2023-10-16T08:39:42.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.463+0000 7f1ec667d640 1 -- 172.21.15.40:0/2258048981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec01529a0 msgr2=0x7f1ec00c36f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:42.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.463+0000 7f1ec667d640 1 --2- 172.21.15.40:0/2258048981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec01529a0 0x7f1ec00c36f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f1eb001c960 tx=0x7f1ea8005800 comp rx=0 tx=0).stop 2023-10-16T08:39:42.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.463+0000 7f1ec667d640 1 -- 172.21.15.40:0/2258048981 shutdown_connections 2023-10-16T08:39:42.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.463+0000 7f1ec667d640 1 --2- 172.21.15.40:0/2258048981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ec01594a0 0x7f1ec0159930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:42.474 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.463+0000 7f1ec667d640 1 --2- 172.21.15.40:0/2258048981 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1eac0664d0 0x7f1eac068990 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:42.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.463+0000 7f1ec667d640 1 --2- 172.21.15.40:0/2258048981 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1ec01529a0 0x7f1ec00c36f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:42.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.463+0000 7f1ec667d640 1 --2- 172.21.15.40:0/2258048981 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1ea80047d0 0x7f1ec014fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:42.475 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.463+0000 7f1ec667d640 1 -- 172.21.15.40:0/2258048981 >> 172.21.15.40:0/2258048981 conn(0x7f1ec00ce960 msgr2=0x7f1ec00ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:42.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.463+0000 7f1ec667d640 1 -- 172.21.15.40:0/2258048981 shutdown_connections 2023-10-16T08:39:42.476 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.463+0000 7f1ec667d640 1 -- 172.21.15.40:0/2258048981 wait complete. 2023-10-16T08:39:42.489 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2023-10-16T08:39:40.957890+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596433,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":218,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":169,"ondisk_log_size":169,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219200,"kb_used_data":3024,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755520,"statfs":{"total":773094113280,"available":772869652480,"internally_reserved":0,"allocated":3096576,"data_stored":1859362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004432"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.964601+0000","last_change":"2023-10-16T08:39:26.963087+0000","last_active":"2023-10-16T08:39:26.964601+0000","last_peered":"2023-10-16T08:39:26.964601+0000","last_clean":"2023-10-16T08:39:26.964601+0000","last_became_active":"2023-10-16T08:39:26.962446+0000","last_became_peered":"2023-10-16T08:39:26.962446+0000","last_unstale":"2023-10-16T08:39:26.964601+0000","last_undegraded":"2023-10-16T08:39:26.964601+0000","last_fullsized":"2023-10-16T08:39:26.964601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:33.190218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945446+0000","last_change":"2023-10-16T08:38:45.657659+0000","last_active":"2023-10-16T08:39:25.945446+0000","last_peered":"2023-10-16T08:39:25.945446+0000","last_clean":"2023-10-16T08:39:25.945446+0000","last_became_active":"2023-10-16T08:38:45.656884+0000","last_became_peered":"2023-10-16T08:38:45.656884+0000","last_unstale":"2023-10-16T08:39:25.945446+0000","last_undegraded":"2023-10-16T08:39:25.945446+0000","last_fullsized":"2023-10-16T08:39:25.945446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402116+0000","last_change":"2023-10-16T08:38:45.630125+0000","last_active":"2023-10-16T08:39:26.402116+0000","last_peered":"2023-10-16T08:39:26.402116+0000","last_clean":"2023-10-16T08:39:26.402116+0000","last_became_active":"2023-10-16T08:38:45.629297+0000","last_became_peered":"2023-10-16T08:38:45.629297+0000","last_unstale":"2023-10-16T08:39:26.402116+0000","last_undegraded":"2023-10-16T08:39:26.402116+0000","last_fullsized":"2023-10-16T08:39:26.402116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951941+0000","last_change":"2023-10-16T08:38:45.675996+0000","last_active":"2023-10-16T08:39:25.951941+0000","last_peered":"2023-10-16T08:39:25.951941+0000","last_clean":"2023-10-16T08:39:25.951941+0000","last_became_active":"2023-10-16T08:38:45.675327+0000","last_became_peered":"2023-10-16T08:38:45.675327+0000","last_unstale":"2023-10-16T08:39:25.951941+0000","last_undegraded":"2023-10-16T08:39:25.951941+0000","last_fullsized":"2023-10-16T08:39:25.951941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:33.190218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955358+0000","last_change":"2023-10-16T08:38:45.653550+0000","last_active":"2023-10-16T08:39:25.955358+0000","last_peered":"2023-10-16T08:39:25.955358+0000","last_clean":"2023-10-16T08:39:25.955358+0000","last_became_active":"2023-10-16T08:38:45.652808+0000","last_became_peered":"2023-10-16T08:38:45.652808+0000","last_unstale":"2023-10-16T08:39:25.955358+0000","last_undegraded":"2023-10-16T08:39:25.955358+0000","last_fullsized":"2023-10-16T08:39:25.955358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.950945+0000","last_change":"2023-10-16T08:38:45.675086+0000","last_active":"2023-10-16T08:39:25.950945+0000","last_peered":"2023-10-16T08:39:25.950945+0000","last_clean":"2023-10-16T08:39:25.950945+0000","last_became_active":"2023-10-16T08:38:45.671146+0000","last_became_peered":"2023-10-16T08:38:45.671146+0000","last_unstale":"2023-10-16T08:39:25.950945+0000","last_undegraded":"2023-10-16T08:39:25.950945+0000","last_fullsized":"2023-10-16T08:39:25.950945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954769+0000","last_change":"2023-10-16T08:38:45.652595+0000","last_active":"2023-10-16T08:39:25.954769+0000","last_peered":"2023-10-16T08:39:25.954769+0000","last_clean":"2023-10-16T08:39:25.954769+0000","last_became_active":"2023-10-16T08:38:45.651848+0000","last_became_peered":"2023-10-16T08:38:45.651848+0000","last_unstale":"2023-10-16T08:39:25.954769+0000","last_undegraded":"2023-10-16T08:39:25.954769+0000","last_fullsized":"2023-10-16T08:39:25.954769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:13.453321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955228+0000","last_change":"2023-10-16T08:38:45.668858+0000","last_active":"2023-10-16T08:39:25.955228+0000","last_peered":"2023-10-16T08:39:25.955228+0000","last_clean":"2023-10-16T08:39:25.955228+0000","last_became_active":"2023-10-16T08:38:45.664702+0000","last_became_peered":"2023-10-16T08:38:45.664702+0000","last_unstale":"2023-10-16T08:39:25.955228+0000","last_undegraded":"2023-10-16T08:39:25.955228+0000","last_fullsized":"2023-10-16T08:39:25.955228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955879+0000","last_change":"2023-10-16T08:38:45.667037+0000","last_active":"2023-10-16T08:39:25.955879+0000","last_peered":"2023-10-16T08:39:25.955879+0000","last_clean":"2023-10-16T08:39:25.955879+0000","last_became_active":"2023-10-16T08:38:45.666474+0000","last_became_peered":"2023-10-16T08:38:45.666474+0000","last_unstale":"2023-10-16T08:39:25.955879+0000","last_undegraded":"2023-10-16T08:39:25.955879+0000","last_fullsized":"2023-10-16T08:39:25.955879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:11:02.634006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949980+0000","last_change":"2023-10-16T08:38:45.669859+0000","last_active":"2023-10-16T08:39:25.949980+0000","last_peered":"2023-10-16T08:39:25.949980+0000","last_clean":"2023-10-16T08:39:25.949980+0000","last_became_active":"2023-10-16T08:38:45.668278+0000","last_became_peered":"2023-10-16T08:38:45.668278+0000","last_unstale":"2023-10-16T08:39:25.949980+0000","last_undegraded":"2023-10-16T08:39:25.949980+0000","last_fullsized":"2023-10-16T08:39:25.949980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949617+0000","last_change":"2023-10-16T08:38:45.664567+0000","last_active":"2023-10-16T08:39:25.949617+0000","last_peered":"2023-10-16T08:39:25.949617+0000","last_clean":"2023-10-16T08:39:25.949617+0000","last_became_active":"2023-10-16T08:38:45.663813+0000","last_became_peered":"2023-10-16T08:38:45.663813+0000","last_unstale":"2023-10-16T08:39:25.949617+0000","last_undegraded":"2023-10-16T08:39:25.949617+0000","last_fullsized":"2023-10-16T08:39:25.949617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403718+0000","last_change":"2023-10-16T08:38:45.675925+0000","last_active":"2023-10-16T08:39:26.403718+0000","last_peered":"2023-10-16T08:39:26.403718+0000","last_clean":"2023-10-16T08:39:26.403718+0000","last_became_active":"2023-10-16T08:38:45.675320+0000","last_became_peered":"2023-10-16T08:38:45.675320+0000","last_unstale":"2023-10-16T08:39:26.403718+0000","last_undegraded":"2023-10-16T08:39:26.403718+0000","last_fullsized":"2023-10-16T08:39:26.403718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:11.991837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956028+0000","last_change":"2023-10-16T08:38:45.668056+0000","last_active":"2023-10-16T08:39:25.956028+0000","last_peered":"2023-10-16T08:39:25.956028+0000","last_clean":"2023-10-16T08:39:25.956028+0000","last_became_active":"2023-10-16T08:38:45.667428+0000","last_became_peered":"2023-10-16T08:38:45.667428+0000","last_unstale":"2023-10-16T08:39:25.956028+0000","last_undegraded":"2023-10-16T08:39:25.956028+0000","last_fullsized":"2023-10-16T08:39:25.956028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.401718+0000","last_change":"2023-10-16T08:38:45.637518+0000","last_active":"2023-10-16T08:39:26.401718+0000","last_peered":"2023-10-16T08:39:26.401718+0000","last_clean":"2023-10-16T08:39:26.401718+0000","last_became_active":"2023-10-16T08:38:45.636785+0000","last_became_peered":"2023-10-16T08:38:45.636785+0000","last_unstale":"2023-10-16T08:39:26.401718+0000","last_undegraded":"2023-10-16T08:39:26.401718+0000","last_fullsized":"2023-10-16T08:39:26.401718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:11.870171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.944872+0000","last_change":"2023-10-16T08:38:45.656433+0000","last_active":"2023-10-16T08:39:25.944872+0000","last_peered":"2023-10-16T08:39:25.944872+0000","last_clean":"2023-10-16T08:39:25.944872+0000","last_became_active":"2023-10-16T08:38:45.655395+0000","last_became_peered":"2023-10-16T08:38:45.655395+0000","last_unstale":"2023-10-16T08:39:25.944872+0000","last_undegraded":"2023-10-16T08:39:25.944872+0000","last_fullsized":"2023-10-16T08:39:25.944872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.950540+0000","last_change":"2023-10-16T08:38:45.664192+0000","last_active":"2023-10-16T08:39:25.950540+0000","last_peered":"2023-10-16T08:39:25.950540+0000","last_clean":"2023-10-16T08:39:25.950540+0000","last_became_active":"2023-10-16T08:38:45.663312+0000","last_became_peered":"2023-10-16T08:38:45.663312+0000","last_unstale":"2023-10-16T08:39:25.950540+0000","last_undegraded":"2023-10-16T08:39:25.950540+0000","last_fullsized":"2023-10-16T08:39:25.950540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.016191+0000","last_change":"2023-10-16T08:39:27.013093+0000","last_active":"2023-10-16T08:39:27.016191+0000","last_peered":"2023-10-16T08:39:27.016191+0000","last_clean":"2023-10-16T08:39:27.016191+0000","last_became_active":"2023-10-16T08:39:27.012068+0000","last_became_peered":"2023-10-16T08:39:27.012068+0000","last_unstale":"2023-10-16T08:39:27.016191+0000","last_undegraded":"2023-10-16T08:39:27.016191+0000","last_fullsized":"2023-10-16T08:39:27.016191+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945605+0000","last_change":"2023-10-16T08:38:45.631488+0000","last_active":"2023-10-16T08:39:25.945605+0000","last_peered":"2023-10-16T08:39:25.945605+0000","last_clean":"2023-10-16T08:39:25.945605+0000","last_became_active":"2023-10-16T08:38:45.630641+0000","last_became_peered":"2023-10-16T08:38:45.630641+0000","last_unstale":"2023-10-16T08:39:25.945605+0000","last_undegraded":"2023-10-16T08:39:25.945605+0000","last_fullsized":"2023-10-16T08:39:25.945605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:11.991837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955988+0000","last_change":"2023-10-16T08:38:45.650087+0000","last_active":"2023-10-16T08:39:25.955988+0000","last_peered":"2023-10-16T08:39:25.955988+0000","last_clean":"2023-10-16T08:39:25.955988+0000","last_became_active":"2023-10-16T08:38:45.649475+0000","last_became_peered":"2023-10-16T08:38:45.649475+0000","last_unstale":"2023-10-16T08:39:25.955988+0000","last_undegraded":"2023-10-16T08:39:25.955988+0000","last_fullsized":"2023-10-16T08:39:25.955988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:26.292137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949532+0000","last_change":"2023-10-16T08:38:45.667336+0000","last_active":"2023-10-16T08:39:25.949532+0000","last_peered":"2023-10-16T08:39:25.949532+0000","last_clean":"2023-10-16T08:39:25.949532+0000","last_became_active":"2023-10-16T08:38:45.665291+0000","last_became_peered":"2023-10-16T08:38:45.665291+0000","last_unstale":"2023-10-16T08:39:25.949532+0000","last_undegraded":"2023-10-16T08:39:25.949532+0000","last_fullsized":"2023-10-16T08:39:25.949532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402191+0000","last_change":"2023-10-16T08:38:45.654036+0000","last_active":"2023-10-16T08:39:26.402191+0000","last_peered":"2023-10-16T08:39:26.402191+0000","last_clean":"2023-10-16T08:39:26.402191+0000","last_became_active":"2023-10-16T08:38:45.652412+0000","last_became_peered":"2023-10-16T08:38:45.652412+0000","last_unstale":"2023-10-16T08:39:26.402191+0000","last_undegraded":"2023-10-16T08:39:26.402191+0000","last_fullsized":"2023-10-16T08:39:26.402191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951156+0000","last_change":"2023-10-16T08:38:45.677644+0000","last_active":"2023-10-16T08:39:25.951156+0000","last_peered":"2023-10-16T08:39:25.951156+0000","last_clean":"2023-10-16T08:39:25.951156+0000","last_became_active":"2023-10-16T08:38:45.676844+0000","last_became_peered":"2023-10-16T08:38:45.676844+0000","last_unstale":"2023-10-16T08:39:25.951156+0000","last_undegraded":"2023-10-16T08:39:25.951156+0000","last_fullsized":"2023-10-16T08:39:25.951156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955256+0000","last_change":"2023-10-16T08:38:45.663771+0000","last_active":"2023-10-16T08:39:25.955256+0000","last_peered":"2023-10-16T08:39:25.955256+0000","last_clean":"2023-10-16T08:39:25.955256+0000","last_became_active":"2023-10-16T08:38:45.662944+0000","last_became_peered":"2023-10-16T08:38:45.662944+0000","last_unstale":"2023-10-16T08:39:25.955256+0000","last_undegraded":"2023-10-16T08:39:25.955256+0000","last_fullsized":"2023-10-16T08:39:25.955256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.950669+0000","last_change":"2023-10-16T08:38:45.667398+0000","last_active":"2023-10-16T08:39:25.950669+0000","last_peered":"2023-10-16T08:39:25.950669+0000","last_clean":"2023-10-16T08:39:25.950669+0000","last_became_active":"2023-10-16T08:38:45.666641+0000","last_became_peered":"2023-10-16T08:38:45.666641+0000","last_unstale":"2023-10-16T08:39:25.950669+0000","last_undegraded":"2023-10-16T08:39:25.950669+0000","last_fullsized":"2023-10-16T08:39:25.950669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955685+0000","last_change":"2023-10-16T08:38:45.670526+0000","last_active":"2023-10-16T08:39:25.955685+0000","last_peered":"2023-10-16T08:39:25.955685+0000","last_clean":"2023-10-16T08:39:25.955685+0000","last_became_active":"2023-10-16T08:38:45.669734+0000","last_became_peered":"2023-10-16T08:38:45.669734+0000","last_unstale":"2023-10-16T08:39:25.955685+0000","last_undegraded":"2023-10-16T08:39:25.955685+0000","last_fullsized":"2023-10-16T08:39:25.955685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949999+0000","last_change":"2023-10-16T08:38:45.670429+0000","last_active":"2023-10-16T08:39:25.949999+0000","last_peered":"2023-10-16T08:39:25.949999+0000","last_clean":"2023-10-16T08:39:25.949999+0000","last_became_active":"2023-10-16T08:38:45.665972+0000","last_became_peered":"2023-10-16T08:38:45.665972+0000","last_unstale":"2023-10-16T08:39:25.949999+0000","last_undegraded":"2023-10-16T08:39:25.949999+0000","last_fullsized":"2023-10-16T08:39:25.949999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945074+0000","last_change":"2023-10-16T08:38:45.667047+0000","last_active":"2023-10-16T08:39:25.945074+0000","last_peered":"2023-10-16T08:39:25.945074+0000","last_clean":"2023-10-16T08:39:25.945074+0000","last_became_active":"2023-10-16T08:38:45.666403+0000","last_became_peered":"2023-10-16T08:38:45.666403+0000","last_unstale":"2023-10-16T08:39:25.945074+0000","last_undegraded":"2023-10-16T08:39:25.945074+0000","last_fullsized":"2023-10-16T08:39:25.945074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956047+0000","last_change":"2023-10-16T08:38:45.668007+0000","last_active":"2023-10-16T08:39:25.956047+0000","last_peered":"2023-10-16T08:39:25.956047+0000","last_clean":"2023-10-16T08:39:25.956047+0000","last_became_active":"2023-10-16T08:38:45.664572+0000","last_became_peered":"2023-10-16T08:38:45.664572+0000","last_unstale":"2023-10-16T08:39:25.956047+0000","last_undegraded":"2023-10-16T08:39:25.956047+0000","last_fullsized":"2023-10-16T08:39:25.956047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402369+0000","last_change":"2023-10-16T08:38:45.667378+0000","last_active":"2023-10-16T08:39:26.402369+0000","last_peered":"2023-10-16T08:39:26.402369+0000","last_clean":"2023-10-16T08:39:26.402369+0000","last_became_active":"2023-10-16T08:38:45.666547+0000","last_became_peered":"2023-10-16T08:38:45.666547+0000","last_unstale":"2023-10-16T08:39:26.402369+0000","last_undegraded":"2023-10-16T08:39:26.402369+0000","last_fullsized":"2023-10-16T08:39:26.402369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.013174+0000","last_change":"2023-10-16T08:39:27.010902+0000","last_active":"2023-10-16T08:39:27.013174+0000","last_peered":"2023-10-16T08:39:27.013174+0000","last_clean":"2023-10-16T08:39:27.013174+0000","last_became_active":"2023-10-16T08:39:27.009983+0000","last_became_peered":"2023-10-16T08:39:27.009983+0000","last_unstale":"2023-10-16T08:39:27.013174+0000","last_undegraded":"2023-10-16T08:39:27.013174+0000","last_fullsized":"2023-10-16T08:39:27.013174+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.014469+0000","last_change":"2023-10-16T08:39:27.014469+0000","last_active":"2023-10-16T08:39:27.014469+0000","last_peered":"2023-10-16T08:39:27.014469+0000","last_clean":"2023-10-16T08:39:27.014469+0000","last_became_active":"2023-10-16T08:39:27.013658+0000","last_became_peered":"2023-10-16T08:39:27.013658+0000","last_unstale":"2023-10-16T08:39:27.014469+0000","last_undegraded":"2023-10-16T08:39:27.014469+0000","last_fullsized":"2023-10-16T08:39:27.014469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949746+0000","last_change":"2023-10-16T08:38:45.638183+0000","last_active":"2023-10-16T08:39:25.949746+0000","last_peered":"2023-10-16T08:39:25.949746+0000","last_clean":"2023-10-16T08:39:25.949746+0000","last_became_active":"2023-10-16T08:38:45.636762+0000","last_became_peered":"2023-10-16T08:38:45.636762+0000","last_unstale":"2023-10-16T08:39:25.949746+0000","last_undegraded":"2023-10-16T08:39:25.949746+0000","last_fullsized":"2023-10-16T08:39:25.949746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403351+0000","last_change":"2023-10-16T08:38:45.659464+0000","last_active":"2023-10-16T08:39:26.403351+0000","last_peered":"2023-10-16T08:39:26.403351+0000","last_clean":"2023-10-16T08:39:26.403351+0000","last_became_active":"2023-10-16T08:38:45.658808+0000","last_became_peered":"2023-10-16T08:38:45.658808+0000","last_unstale":"2023-10-16T08:39:26.403351+0000","last_undegraded":"2023-10-16T08:39:26.403351+0000","last_fullsized":"2023-10-16T08:39:26.403351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:25.386221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956227+0000","last_change":"2023-10-16T08:38:44.644122+0000","last_active":"2023-10-16T08:39:25.956227+0000","last_peered":"2023-10-16T08:39:25.956227+0000","last_clean":"2023-10-16T08:39:25.956227+0000","last_became_active":"2023-10-16T08:38:44.643614+0000","last_became_peered":"2023-10-16T08:38:44.643614+0000","last_unstale":"2023-10-16T08:39:25.956227+0000","last_undegraded":"2023-10-16T08:39:25.956227+0000","last_fullsized":"2023-10-16T08:39:25.956227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402402+0000","last_change":"2023-10-16T08:38:45.659854+0000","last_active":"2023-10-16T08:39:26.402402+0000","last_peered":"2023-10-16T08:39:26.402402+0000","last_clean":"2023-10-16T08:39:26.402402+0000","last_became_active":"2023-10-16T08:38:45.658409+0000","last_became_peered":"2023-10-16T08:38:45.658409+0000","last_unstale":"2023-10-16T08:39:26.402402+0000","last_undegraded":"2023-10-16T08:39:26.402402+0000","last_fullsized":"2023-10-16T08:39:26.402402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949532+0000","last_change":"2023-10-16T08:38:44.658819+0000","last_active":"2023-10-16T08:39:25.949532+0000","last_peered":"2023-10-16T08:39:25.949532+0000","last_clean":"2023-10-16T08:39:25.949532+0000","last_became_active":"2023-10-16T08:38:44.658257+0000","last_became_peered":"2023-10-16T08:38:44.658257+0000","last_unstale":"2023-10-16T08:39:25.949532+0000","last_undegraded":"2023-10-16T08:39:25.949532+0000","last_fullsized":"2023-10-16T08:39:25.949532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:54.848445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955337+0000","last_change":"2023-10-16T08:38:44.647536+0000","last_active":"2023-10-16T08:39:25.955337+0000","last_peered":"2023-10-16T08:39:25.955337+0000","last_clean":"2023-10-16T08:39:25.955337+0000","last_became_active":"2023-10-16T08:38:44.645931+0000","last_became_peered":"2023-10-16T08:38:44.645931+0000","last_unstale":"2023-10-16T08:39:25.955337+0000","last_undegraded":"2023-10-16T08:39:25.955337+0000","last_fullsized":"2023-10-16T08:39:25.955337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955491+0000","last_change":"2023-10-16T08:38:45.662855+0000","last_active":"2023-10-16T08:39:25.955491+0000","last_peered":"2023-10-16T08:39:25.955491+0000","last_clean":"2023-10-16T08:39:25.955491+0000","last_became_active":"2023-10-16T08:38:45.662072+0000","last_became_peered":"2023-10-16T08:38:45.662072+0000","last_unstale":"2023-10-16T08:39:25.955491+0000","last_undegraded":"2023-10-16T08:39:25.955491+0000","last_fullsized":"2023-10-16T08:39:25.955491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.950636+0000","last_change":"2023-10-16T08:38:44.657651+0000","last_active":"2023-10-16T08:39:25.950636+0000","last_peered":"2023-10-16T08:39:25.950636+0000","last_clean":"2023-10-16T08:39:25.950636+0000","last_became_active":"2023-10-16T08:38:44.657065+0000","last_became_peered":"2023-10-16T08:38:44.657065+0000","last_unstale":"2023-10-16T08:39:25.950636+0000","last_undegraded":"2023-10-16T08:39:25.950636+0000","last_fullsized":"2023-10-16T08:39:25.950636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.944769+0000","last_change":"2023-10-16T08:38:45.668048+0000","last_active":"2023-10-16T08:39:25.944769+0000","last_peered":"2023-10-16T08:39:25.944769+0000","last_clean":"2023-10-16T08:39:25.944769+0000","last_became_active":"2023-10-16T08:38:45.667413+0000","last_became_peered":"2023-10-16T08:38:45.667413+0000","last_unstale":"2023-10-16T08:39:25.944769+0000","last_undegraded":"2023-10-16T08:39:25.944769+0000","last_fullsized":"2023-10-16T08:39:25.944769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949842+0000","last_change":"2023-10-16T08:38:44.663188+0000","last_active":"2023-10-16T08:39:25.949842+0000","last_peered":"2023-10-16T08:39:25.949842+0000","last_clean":"2023-10-16T08:39:25.949842+0000","last_became_active":"2023-10-16T08:38:44.662383+0000","last_became_peered":"2023-10-16T08:38:44.662383+0000","last_unstale":"2023-10-16T08:39:25.949842+0000","last_undegraded":"2023-10-16T08:39:25.949842+0000","last_fullsized":"2023-10-16T08:39:25.949842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:24.375366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.953576+0000","last_change":"2023-10-16T08:38:45.679271+0000","last_active":"2023-10-16T08:39:25.953576+0000","last_peered":"2023-10-16T08:39:25.953576+0000","last_clean":"2023-10-16T08:39:25.953576+0000","last_became_active":"2023-10-16T08:38:45.678050+0000","last_became_peered":"2023-10-16T08:38:45.678050+0000","last_unstale":"2023-10-16T08:39:25.953576+0000","last_undegraded":"2023-10-16T08:39:25.953576+0000","last_fullsized":"2023-10-16T08:39:25.953576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.953752+0000","last_change":"2023-10-16T08:38:44.653478+0000","last_active":"2023-10-16T08:39:25.953752+0000","last_peered":"2023-10-16T08:39:25.953752+0000","last_clean":"2023-10-16T08:39:25.953752+0000","last_became_active":"2023-10-16T08:38:44.652618+0000","last_became_peered":"2023-10-16T08:38:44.652618+0000","last_unstale":"2023-10-16T08:39:25.953752+0000","last_undegraded":"2023-10-16T08:39:25.953752+0000","last_fullsized":"2023-10-16T08:39:25.953752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402370+0000","last_change":"2023-10-16T08:38:45.656603+0000","last_active":"2023-10-16T08:39:26.402370+0000","last_peered":"2023-10-16T08:39:26.402370+0000","last_clean":"2023-10-16T08:39:26.402370+0000","last_became_active":"2023-10-16T08:38:45.655213+0000","last_became_peered":"2023-10-16T08:38:45.655213+0000","last_unstale":"2023-10-16T08:39:26.402370+0000","last_undegraded":"2023-10-16T08:39:26.402370+0000","last_fullsized":"2023-10-16T08:39:26.402370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945084+0000","last_change":"2023-10-16T08:38:44.651637+0000","last_active":"2023-10-16T08:39:25.945084+0000","last_peered":"2023-10-16T08:39:25.945084+0000","last_clean":"2023-10-16T08:39:25.945084+0000","last_became_active":"2023-10-16T08:38:44.651359+0000","last_became_peered":"2023-10-16T08:38:44.651359+0000","last_unstale":"2023-10-16T08:39:25.945084+0000","last_undegraded":"2023-10-16T08:39:25.945084+0000","last_fullsized":"2023-10-16T08:39:25.945084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:10.859316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403856+0000","last_change":"2023-10-16T08:38:45.661741+0000","last_active":"2023-10-16T08:39:26.403856+0000","last_peered":"2023-10-16T08:39:26.403856+0000","last_clean":"2023-10-16T08:39:26.403856+0000","last_became_active":"2023-10-16T08:38:45.660611+0000","last_became_peered":"2023-10-16T08:38:45.660611+0000","last_unstale":"2023-10-16T08:39:26.403856+0000","last_undegraded":"2023-10-16T08:39:26.403856+0000","last_fullsized":"2023-10-16T08:39:26.403856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:55.859300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'18","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:35.375472+0000","last_change":"2023-10-16T08:38:44.647593+0000","last_active":"2023-10-16T08:39:35.375472+0000","last_peered":"2023-10-16T08:39:35.375472+0000","last_clean":"2023-10-16T08:39:35.375472+0000","last_became_active":"2023-10-16T08:38:44.647001+0000","last_became_peered":"2023-10-16T08:38:44.647001+0000","last_unstale":"2023-10-16T08:39:35.375472+0000","last_undegraded":"2023-10-16T08:39:35.375472+0000","last_fullsized":"2023-10-16T08:39:35.375472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5267,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955477+0000","last_change":"2023-10-16T08:38:45.668629+0000","last_active":"2023-10-16T08:39:25.955477+0000","last_peered":"2023-10-16T08:39:25.955477+0000","last_clean":"2023-10-16T08:39:25.955477+0000","last_became_active":"2023-10-16T08:38:45.667334+0000","last_became_peered":"2023-10-16T08:38:45.667334+0000","last_unstale":"2023-10-16T08:39:25.955477+0000","last_undegraded":"2023-10-16T08:39:25.955477+0000","last_fullsized":"2023-10-16T08:39:25.955477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402610+0000","last_change":"2023-10-16T08:38:44.655832+0000","last_active":"2023-10-16T08:39:26.402610+0000","last_peered":"2023-10-16T08:39:26.402610+0000","last_clean":"2023-10-16T08:39:26.402610+0000","last_became_active":"2023-10-16T08:38:44.655469+0000","last_became_peered":"2023-10-16T08:38:44.655469+0000","last_unstale":"2023-10-16T08:39:26.402610+0000","last_undegraded":"2023-10-16T08:39:26.402610+0000","last_fullsized":"2023-10-16T08:39:26.402610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:19.768762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.401855+0000","last_change":"2023-10-16T08:38:45.653688+0000","last_active":"2023-10-16T08:39:26.401855+0000","last_peered":"2023-10-16T08:39:26.401855+0000","last_clean":"2023-10-16T08:39:26.401855+0000","last_became_active":"2023-10-16T08:38:45.650563+0000","last_became_peered":"2023-10-16T08:38:45.650563+0000","last_unstale":"2023-10-16T08:39:26.401855+0000","last_undegraded":"2023-10-16T08:39:26.401855+0000","last_fullsized":"2023-10-16T08:39:26.401855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:29.539285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951818+0000","last_change":"2023-10-16T08:38:44.662816+0000","last_active":"2023-10-16T08:39:25.951818+0000","last_peered":"2023-10-16T08:39:25.951818+0000","last_clean":"2023-10-16T08:39:25.951818+0000","last_became_active":"2023-10-16T08:38:44.662110+0000","last_became_peered":"2023-10-16T08:38:44.662110+0000","last_unstale":"2023-10-16T08:39:25.951818+0000","last_undegraded":"2023-10-16T08:39:25.951818+0000","last_fullsized":"2023-10-16T08:39:25.951818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:00.201724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955811+0000","last_change":"2023-10-16T08:38:45.662166+0000","last_active":"2023-10-16T08:39:25.955811+0000","last_peered":"2023-10-16T08:39:25.955811+0000","last_clean":"2023-10-16T08:39:25.955811+0000","last_became_active":"2023-10-16T08:38:45.661595+0000","last_became_peered":"2023-10-16T08:38:45.661595+0000","last_unstale":"2023-10-16T08:39:25.955811+0000","last_undegraded":"2023-10-16T08:39:25.955811+0000","last_fullsized":"2023-10-16T08:39:25.955811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:02:39.840175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949503+0000","last_change":"2023-10-16T08:38:44.657766+0000","last_active":"2023-10-16T08:39:25.949503+0000","last_peered":"2023-10-16T08:39:25.949503+0000","last_clean":"2023-10-16T08:39:25.949503+0000","last_became_active":"2023-10-16T08:38:44.657153+0000","last_became_peered":"2023-10-16T08:38:44.657153+0000","last_unstale":"2023-10-16T08:39:25.949503+0000","last_undegraded":"2023-10-16T08:39:25.949503+0000","last_fullsized":"2023-10-16T08:39:25.949503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949501+0000","last_change":"2023-10-16T08:38:45.637591+0000","last_active":"2023-10-16T08:39:25.949501+0000","last_peered":"2023-10-16T08:39:25.949501+0000","last_clean":"2023-10-16T08:39:25.949501+0000","last_became_active":"2023-10-16T08:38:45.635633+0000","last_became_peered":"2023-10-16T08:38:45.635633+0000","last_unstale":"2023-10-16T08:39:25.949501+0000","last_undegraded":"2023-10-16T08:39:25.949501+0000","last_fullsized":"2023-10-16T08:39:25.949501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.946160+0000","last_change":"2023-10-16T08:38:44.638297+0000","last_active":"2023-10-16T08:39:25.946160+0000","last_peered":"2023-10-16T08:39:25.946160+0000","last_clean":"2023-10-16T08:39:25.946160+0000","last_became_active":"2023-10-16T08:38:44.637640+0000","last_became_peered":"2023-10-16T08:38:44.637640+0000","last_unstale":"2023-10-16T08:39:25.946160+0000","last_undegraded":"2023-10-16T08:39:25.946160+0000","last_fullsized":"2023-10-16T08:39:25.946160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:16.003943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954941+0000","last_change":"2023-10-16T08:38:45.657286+0000","last_active":"2023-10-16T08:39:25.954941+0000","last_peered":"2023-10-16T08:39:25.954941+0000","last_clean":"2023-10-16T08:39:25.954941+0000","last_became_active":"2023-10-16T08:38:45.656651+0000","last_became_peered":"2023-10-16T08:38:45.656651+0000","last_unstale":"2023-10-16T08:39:25.954941+0000","last_undegraded":"2023-10-16T08:39:25.954941+0000","last_fullsized":"2023-10-16T08:39:25.954941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:28.052483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403973+0000","last_change":"2023-10-16T08:38:44.648434+0000","last_active":"2023-10-16T08:39:26.403973+0000","last_peered":"2023-10-16T08:39:26.403973+0000","last_clean":"2023-10-16T08:39:26.403973+0000","last_became_active":"2023-10-16T08:38:44.647887+0000","last_became_peered":"2023-10-16T08:38:44.647887+0000","last_unstale":"2023-10-16T08:39:26.403973+0000","last_undegraded":"2023-10-16T08:39:26.403973+0000","last_fullsized":"2023-10-16T08:39:26.403973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954988+0000","last_change":"2023-10-16T08:38:45.661965+0000","last_active":"2023-10-16T08:39:25.954988+0000","last_peered":"2023-10-16T08:39:25.954988+0000","last_clean":"2023-10-16T08:39:25.954988+0000","last_became_active":"2023-10-16T08:38:45.656750+0000","last_became_peered":"2023-10-16T08:38:45.656750+0000","last_unstale":"2023-10-16T08:39:25.954988+0000","last_undegraded":"2023-10-16T08:39:25.954988+0000","last_fullsized":"2023-10-16T08:39:25.954988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945809+0000","last_change":"2023-10-16T08:38:44.647524+0000","last_active":"2023-10-16T08:39:25.945809+0000","last_peered":"2023-10-16T08:39:25.945809+0000","last_clean":"2023-10-16T08:39:25.945809+0000","last_became_active":"2023-10-16T08:38:44.646042+0000","last_became_peered":"2023-10-16T08:38:44.646042+0000","last_unstale":"2023-10-16T08:39:25.945809+0000","last_undegraded":"2023-10-16T08:39:25.945809+0000","last_fullsized":"2023-10-16T08:39:25.945809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:24.375366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955916+0000","last_change":"2023-10-16T08:38:45.670969+0000","last_active":"2023-10-16T08:39:25.955916+0000","last_peered":"2023-10-16T08:39:25.955916+0000","last_clean":"2023-10-16T08:39:25.955916+0000","last_became_active":"2023-10-16T08:38:45.670415+0000","last_became_peered":"2023-10-16T08:38:45.670415+0000","last_unstale":"2023-10-16T08:39:25.955916+0000","last_undegraded":"2023-10-16T08:39:25.955916+0000","last_fullsized":"2023-10-16T08:39:25.955916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:15:00.472153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945149+0000","last_change":"2023-10-16T08:38:44.655744+0000","last_active":"2023-10-16T08:39:25.945149+0000","last_peered":"2023-10-16T08:39:25.945149+0000","last_clean":"2023-10-16T08:39:25.945149+0000","last_became_active":"2023-10-16T08:38:44.655260+0000","last_became_peered":"2023-10-16T08:38:44.655260+0000","last_unstale":"2023-10-16T08:39:25.945149+0000","last_undegraded":"2023-10-16T08:39:25.945149+0000","last_fullsized":"2023-10-16T08:39:25.945149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:54.848445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951801+0000","last_change":"2023-10-16T08:38:45.668230+0000","last_active":"2023-10-16T08:39:25.951801+0000","last_peered":"2023-10-16T08:39:25.951801+0000","last_clean":"2023-10-16T08:39:25.951801+0000","last_became_active":"2023-10-16T08:38:45.667627+0000","last_became_peered":"2023-10-16T08:38:45.667627+0000","last_unstale":"2023-10-16T08:39:25.951801+0000","last_undegraded":"2023-10-16T08:39:25.951801+0000","last_fullsized":"2023-10-16T08:39:25.951801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:28.052483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945644+0000","last_change":"2023-10-16T08:38:44.662118+0000","last_active":"2023-10-16T08:39:25.945644+0000","last_peered":"2023-10-16T08:39:25.945644+0000","last_clean":"2023-10-16T08:39:25.945644+0000","last_became_active":"2023-10-16T08:38:44.661736+0000","last_became_peered":"2023-10-16T08:38:44.661736+0000","last_unstale":"2023-10-16T08:39:25.945644+0000","last_undegraded":"2023-10-16T08:39:25.945644+0000","last_fullsized":"2023-10-16T08:39:25.945644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:49:40.967349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955744+0000","last_change":"2023-10-16T08:38:45.667423+0000","last_active":"2023-10-16T08:39:25.955744+0000","last_peered":"2023-10-16T08:39:25.955744+0000","last_clean":"2023-10-16T08:39:25.955744+0000","last_became_active":"2023-10-16T08:38:45.666390+0000","last_became_peered":"2023-10-16T08:38:45.666390+0000","last_unstale":"2023-10-16T08:39:25.955744+0000","last_undegraded":"2023-10-16T08:39:25.955744+0000","last_fullsized":"2023-10-16T08:39:25.955744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:33.190218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":167,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954612+0000","last_change":"2023-10-16T08:38:34.313511+0000","last_active":"2023-10-16T08:39:25.954612+0000","last_peered":"2023-10-16T08:39:25.954612+0000","last_clean":"2023-10-16T08:39:25.954612+0000","last_became_active":"2023-10-16T08:38:34.310944+0000","last_became_peered":"2023-10-16T08:38:34.310944+0000","last_unstale":"2023-10-16T08:39:25.954612+0000","last_undegraded":"2023-10-16T08:39:25.954612+0000","last_fullsized":"2023-10-16T08:39:25.954612+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:33.264692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:33.264692+0000","last_clean_scrub_stamp":"2023-10-16T08:38:33.264692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:50.170461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951158+0000","last_change":"2023-10-16T08:38:44.655223+0000","last_active":"2023-10-16T08:39:25.951158+0000","last_peered":"2023-10-16T08:39:25.951158+0000","last_clean":"2023-10-16T08:39:25.951158+0000","last_became_active":"2023-10-16T08:38:44.654277+0000","last_became_peered":"2023-10-16T08:38:44.654277+0000","last_unstale":"2023-10-16T08:39:25.951158+0000","last_undegraded":"2023-10-16T08:39:25.951158+0000","last_fullsized":"2023-10-16T08:39:25.951158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:03.462433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955641+0000","last_change":"2023-10-16T08:38:45.679868+0000","last_active":"2023-10-16T08:39:25.955641+0000","last_peered":"2023-10-16T08:39:25.955641+0000","last_clean":"2023-10-16T08:39:25.955641+0000","last_became_active":"2023-10-16T08:38:45.679409+0000","last_became_peered":"2023-10-16T08:38:45.679409+0000","last_unstale":"2023-10-16T08:39:25.955641+0000","last_undegraded":"2023-10-16T08:39:25.955641+0000","last_fullsized":"2023-10-16T08:39:25.955641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954528+0000","last_change":"2023-10-16T08:38:44.652152+0000","last_active":"2023-10-16T08:39:25.954528+0000","last_peered":"2023-10-16T08:39:25.954528+0000","last_clean":"2023-10-16T08:39:25.954528+0000","last_became_active":"2023-10-16T08:38:44.651534+0000","last_became_peered":"2023-10-16T08:38:44.651534+0000","last_unstale":"2023-10-16T08:39:25.954528+0000","last_undegraded":"2023-10-16T08:39:25.954528+0000","last_fullsized":"2023-10-16T08:39:25.954528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.401719+0000","last_change":"2023-10-16T08:38:45.651786+0000","last_active":"2023-10-16T08:39:26.401719+0000","last_peered":"2023-10-16T08:39:26.401719+0000","last_clean":"2023-10-16T08:39:26.401719+0000","last_became_active":"2023-10-16T08:38:45.650715+0000","last_became_peered":"2023-10-16T08:38:45.650715+0000","last_unstale":"2023-10-16T08:39:26.401719+0000","last_undegraded":"2023-10-16T08:39:26.401719+0000","last_fullsized":"2023-10-16T08:39:26.401719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:29.799799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.946109+0000","last_change":"2023-10-16T08:38:44.645933+0000","last_active":"2023-10-16T08:39:25.946109+0000","last_peered":"2023-10-16T08:39:25.946109+0000","last_clean":"2023-10-16T08:39:25.946109+0000","last_became_active":"2023-10-16T08:38:44.645451+0000","last_became_peered":"2023-10-16T08:38:44.645451+0000","last_unstale":"2023-10-16T08:39:25.946109+0000","last_undegraded":"2023-10-16T08:39:25.946109+0000","last_fullsized":"2023-10-16T08:39:25.946109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.944780+0000","last_change":"2023-10-16T08:38:45.641499+0000","last_active":"2023-10-16T08:39:25.944780+0000","last_peered":"2023-10-16T08:39:25.944780+0000","last_clean":"2023-10-16T08:39:25.944780+0000","last_became_active":"2023-10-16T08:38:45.640765+0000","last_became_peered":"2023-10-16T08:38:45.640765+0000","last_unstale":"2023-10-16T08:39:25.944780+0000","last_undegraded":"2023-10-16T08:39:25.944780+0000","last_fullsized":"2023-10-16T08:39:25.944780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.952050+0000","last_change":"2023-10-16T08:38:44.655401+0000","last_active":"2023-10-16T08:39:25.952050+0000","last_peered":"2023-10-16T08:39:25.952050+0000","last_clean":"2023-10-16T08:39:25.952050+0000","last_became_active":"2023-10-16T08:38:44.654622+0000","last_became_peered":"2023-10-16T08:38:44.654622+0000","last_unstale":"2023-10-16T08:39:25.952050+0000","last_undegraded":"2023-10-16T08:39:25.952050+0000","last_fullsized":"2023-10-16T08:39:25.952050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:48.705915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403580+0000","last_change":"2023-10-16T08:38:45.659377+0000","last_active":"2023-10-16T08:39:26.403580+0000","last_peered":"2023-10-16T08:39:26.403580+0000","last_clean":"2023-10-16T08:39:26.403580+0000","last_became_active":"2023-10-16T08:38:45.658594+0000","last_became_peered":"2023-10-16T08:38:45.658594+0000","last_unstale":"2023-10-16T08:39:26.403580+0000","last_undegraded":"2023-10-16T08:39:26.403580+0000","last_fullsized":"2023-10-16T08:39:26.403580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:56.738058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945087+0000","last_change":"2023-10-16T08:38:44.634158+0000","last_active":"2023-10-16T08:39:25.945087+0000","last_peered":"2023-10-16T08:39:25.945087+0000","last_clean":"2023-10-16T08:39:25.945087+0000","last_became_active":"2023-10-16T08:38:44.633854+0000","last_became_peered":"2023-10-16T08:38:44.633854+0000","last_unstale":"2023-10-16T08:39:25.945087+0000","last_undegraded":"2023-10-16T08:39:25.945087+0000","last_fullsized":"2023-10-16T08:39:25.945087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:55.727203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.015974+0000","last_change":"2023-10-16T08:39:27.015974+0000","last_active":"2023-10-16T08:39:27.015974+0000","last_peered":"2023-10-16T08:39:27.015974+0000","last_clean":"2023-10-16T08:39:27.015974+0000","last_became_active":"2023-10-16T08:39:27.015266+0000","last_became_peered":"2023-10-16T08:39:27.015266+0000","last_unstale":"2023-10-16T08:39:27.015974+0000","last_undegraded":"2023-10-16T08:39:27.015974+0000","last_fullsized":"2023-10-16T08:39:27.015974+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:41:52.951028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956207+0000","last_change":"2023-10-16T08:38:44.655693+0000","last_active":"2023-10-16T08:39:25.956207+0000","last_peered":"2023-10-16T08:39:25.956207+0000","last_clean":"2023-10-16T08:39:25.956207+0000","last_became_active":"2023-10-16T08:38:44.655001+0000","last_became_peered":"2023-10-16T08:38:44.655001+0000","last_unstale":"2023-10-16T08:39:25.956207+0000","last_undegraded":"2023-10-16T08:39:25.956207+0000","last_fullsized":"2023-10-16T08:39:25.956207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:30.649639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945004+0000","last_change":"2023-10-16T08:38:45.639658+0000","last_active":"2023-10-16T08:39:25.945004+0000","last_peered":"2023-10-16T08:39:25.945004+0000","last_clean":"2023-10-16T08:39:25.945004+0000","last_became_active":"2023-10-16T08:38:45.638702+0000","last_became_peered":"2023-10-16T08:38:45.638702+0000","last_unstale":"2023-10-16T08:39:25.945004+0000","last_undegraded":"2023-10-16T08:39:25.945004+0000","last_fullsized":"2023-10-16T08:39:25.945004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945364+0000","last_change":"2023-10-16T08:38:44.648715+0000","last_active":"2023-10-16T08:39:25.945364+0000","last_peered":"2023-10-16T08:39:25.945364+0000","last_clean":"2023-10-16T08:39:25.945364+0000","last_became_active":"2023-10-16T08:38:44.648312+0000","last_became_peered":"2023-10-16T08:38:44.648312+0000","last_unstale":"2023-10-16T08:39:25.945364+0000","last_undegraded":"2023-10-16T08:39:25.945364+0000","last_fullsized":"2023-10-16T08:39:25.945364+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955768+0000","last_change":"2023-10-16T08:38:45.663402+0000","last_active":"2023-10-16T08:39:25.955768+0000","last_peered":"2023-10-16T08:39:25.955768+0000","last_clean":"2023-10-16T08:39:25.955768+0000","last_became_active":"2023-10-16T08:38:45.662442+0000","last_became_peered":"2023-10-16T08:38:45.662442+0000","last_unstale":"2023-10-16T08:39:25.955768+0000","last_undegraded":"2023-10-16T08:39:25.955768+0000","last_fullsized":"2023-10-16T08:39:25.955768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949981+0000","last_change":"2023-10-16T08:38:44.650015+0000","last_active":"2023-10-16T08:39:25.949981+0000","last_peered":"2023-10-16T08:39:25.949981+0000","last_clean":"2023-10-16T08:39:25.949981+0000","last_became_active":"2023-10-16T08:38:44.648155+0000","last_became_peered":"2023-10-16T08:38:44.648155+0000","last_unstale":"2023-10-16T08:39:25.949981+0000","last_undegraded":"2023-10-16T08:39:25.949981+0000","last_fullsized":"2023-10-16T08:39:25.949981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402266+0000","last_change":"2023-10-16T08:38:45.637568+0000","last_active":"2023-10-16T08:39:26.402266+0000","last_peered":"2023-10-16T08:39:26.402266+0000","last_clean":"2023-10-16T08:39:26.402266+0000","last_became_active":"2023-10-16T08:38:45.636751+0000","last_became_peered":"2023-10-16T08:38:45.636751+0000","last_unstale":"2023-10-16T08:39:26.402266+0000","last_undegraded":"2023-10-16T08:39:26.402266+0000","last_fullsized":"2023-10-16T08:39:26.402266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954444+0000","last_change":"2023-10-16T08:38:44.649257+0000","last_active":"2023-10-16T08:39:25.954444+0000","last_peered":"2023-10-16T08:39:25.954444+0000","last_clean":"2023-10-16T08:39:25.954444+0000","last_became_active":"2023-10-16T08:38:44.646990+0000","last_became_peered":"2023-10-16T08:38:44.646990+0000","last_unstale":"2023-10-16T08:39:25.954444+0000","last_undegraded":"2023-10-16T08:39:25.954444+0000","last_fullsized":"2023-10-16T08:39:25.954444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403442+0000","last_change":"2023-10-16T08:38:45.648228+0000","last_active":"2023-10-16T08:39:26.403442+0000","last_peered":"2023-10-16T08:39:26.403442+0000","last_clean":"2023-10-16T08:39:26.403442+0000","last_became_active":"2023-10-16T08:38:45.647449+0000","last_became_peered":"2023-10-16T08:38:45.647449+0000","last_unstale":"2023-10-16T08:39:26.403442+0000","last_undegraded":"2023-10-16T08:39:26.403442+0000","last_fullsized":"2023-10-16T08:39:26.403442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:20.779617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.944878+0000","last_change":"2023-10-16T08:38:44.631354+0000","last_active":"2023-10-16T08:39:25.944878+0000","last_peered":"2023-10-16T08:39:25.944878+0000","last_clean":"2023-10-16T08:39:25.944878+0000","last_became_active":"2023-10-16T08:38:44.630451+0000","last_became_peered":"2023-10-16T08:38:44.630451+0000","last_unstale":"2023-10-16T08:39:25.944878+0000","last_undegraded":"2023-10-16T08:39:25.944878+0000","last_fullsized":"2023-10-16T08:39:25.944878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:10.980982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955719+0000","last_change":"2023-10-16T08:38:45.663898+0000","last_active":"2023-10-16T08:39:25.955719+0000","last_peered":"2023-10-16T08:39:25.955719+0000","last_clean":"2023-10-16T08:39:25.955719+0000","last_became_active":"2023-10-16T08:38:45.663100+0000","last_became_peered":"2023-10-16T08:38:45.663100+0000","last_unstale":"2023-10-16T08:39:25.955719+0000","last_undegraded":"2023-10-16T08:39:25.955719+0000","last_fullsized":"2023-10-16T08:39:25.955719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945128+0000","last_change":"2023-10-16T08:38:44.646987+0000","last_active":"2023-10-16T08:39:25.945128+0000","last_peered":"2023-10-16T08:39:25.945128+0000","last_clean":"2023-10-16T08:39:25.945128+0000","last_became_active":"2023-10-16T08:38:44.646580+0000","last_became_peered":"2023-10-16T08:38:44.646580+0000","last_unstale":"2023-10-16T08:39:25.945128+0000","last_undegraded":"2023-10-16T08:39:25.945128+0000","last_fullsized":"2023-10-16T08:39:25.945128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956260+0000","last_change":"2023-10-16T08:38:45.662894+0000","last_active":"2023-10-16T08:39:25.956260+0000","last_peered":"2023-10-16T08:39:25.956260+0000","last_clean":"2023-10-16T08:39:25.956260+0000","last_became_active":"2023-10-16T08:38:45.662226+0000","last_became_peered":"2023-10-16T08:38:45.662226+0000","last_unstale":"2023-10-16T08:39:25.956260+0000","last_undegraded":"2023-10-16T08:39:25.956260+0000","last_fullsized":"2023-10-16T08:39:25.956260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955552+0000","last_change":"2023-10-16T08:38:44.660571+0000","last_active":"2023-10-16T08:39:25.955552+0000","last_peered":"2023-10-16T08:39:25.955552+0000","last_clean":"2023-10-16T08:39:25.955552+0000","last_became_active":"2023-10-16T08:38:44.659976+0000","last_became_peered":"2023-10-16T08:38:44.659976+0000","last_unstale":"2023-10-16T08:39:25.955552+0000","last_undegraded":"2023-10-16T08:39:25.955552+0000","last_fullsized":"2023-10-16T08:39:25.955552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949392+0000","last_change":"2023-10-16T08:38:45.665200+0000","last_active":"2023-10-16T08:39:25.949392+0000","last_peered":"2023-10-16T08:39:25.949392+0000","last_clean":"2023-10-16T08:39:25.949392+0000","last_became_active":"2023-10-16T08:38:45.664616+0000","last_became_peered":"2023-10-16T08:38:45.664616+0000","last_unstale":"2023-10-16T08:39:25.949392+0000","last_undegraded":"2023-10-16T08:39:25.949392+0000","last_fullsized":"2023-10-16T08:39:25.949392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.953546+0000","last_change":"2023-10-16T08:38:45.680892+0000","last_active":"2023-10-16T08:39:25.953546+0000","last_peered":"2023-10-16T08:39:25.953546+0000","last_clean":"2023-10-16T08:39:25.953546+0000","last_became_active":"2023-10-16T08:38:45.680231+0000","last_became_peered":"2023-10-16T08:38:45.680231+0000","last_unstale":"2023-10-16T08:39:25.953546+0000","last_undegraded":"2023-10-16T08:39:25.953546+0000","last_fullsized":"2023-10-16T08:39:25.953546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:58:51.599709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956015+0000","last_change":"2023-10-16T08:38:44.664269+0000","last_active":"2023-10-16T08:39:25.956015+0000","last_peered":"2023-10-16T08:39:25.956015+0000","last_clean":"2023-10-16T08:39:25.956015+0000","last_became_active":"2023-10-16T08:38:44.663674+0000","last_became_peered":"2023-10-16T08:38:44.663674+0000","last_unstale":"2023-10-16T08:39:25.956015+0000","last_undegraded":"2023-10-16T08:39:25.956015+0000","last_fullsized":"2023-10-16T08:39:25.956015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951695+0000","last_change":"2023-10-16T08:38:45.680890+0000","last_active":"2023-10-16T08:39:25.951695+0000","last_peered":"2023-10-16T08:39:25.951695+0000","last_clean":"2023-10-16T08:39:25.951695+0000","last_became_active":"2023-10-16T08:38:45.680266+0000","last_became_peered":"2023-10-16T08:38:45.680266+0000","last_unstale":"2023-10-16T08:39:25.951695+0000","last_undegraded":"2023-10-16T08:39:25.951695+0000","last_fullsized":"2023-10-16T08:39:25.951695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:26.292137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955778+0000","last_change":"2023-10-16T08:38:44.653447+0000","last_active":"2023-10-16T08:39:25.955778+0000","last_peered":"2023-10-16T08:39:25.955778+0000","last_clean":"2023-10-16T08:39:25.955778+0000","last_became_active":"2023-10-16T08:38:44.652658+0000","last_became_peered":"2023-10-16T08:38:44.652658+0000","last_unstale":"2023-10-16T08:39:25.955778+0000","last_undegraded":"2023-10-16T08:39:25.955778+0000","last_fullsized":"2023-10-16T08:39:25.955778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:22.350066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402179+0000","last_change":"2023-10-16T08:38:45.665041+0000","last_active":"2023-10-16T08:39:26.402179+0000","last_peered":"2023-10-16T08:39:26.402179+0000","last_clean":"2023-10-16T08:39:26.402179+0000","last_became_active":"2023-10-16T08:38:45.664274+0000","last_became_peered":"2023-10-16T08:38:45.664274+0000","last_unstale":"2023-10-16T08:39:26.402179+0000","last_undegraded":"2023-10-16T08:39:26.402179+0000","last_fullsized":"2023-10-16T08:39:26.402179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955588+0000","last_change":"2023-10-16T08:38:44.645754+0000","last_active":"2023-10-16T08:39:25.955588+0000","last_peered":"2023-10-16T08:39:25.955588+0000","last_clean":"2023-10-16T08:39:25.955588+0000","last_became_active":"2023-10-16T08:38:44.645000+0000","last_became_peered":"2023-10-16T08:38:44.645000+0000","last_unstale":"2023-10-16T08:39:25.955588+0000","last_undegraded":"2023-10-16T08:39:25.955588+0000","last_fullsized":"2023-10-16T08:39:25.955588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955495+0000","last_change":"2023-10-16T08:38:45.664252+0000","last_active":"2023-10-16T08:39:25.955495+0000","last_peered":"2023-10-16T08:39:25.955495+0000","last_clean":"2023-10-16T08:39:25.955495+0000","last_became_active":"2023-10-16T08:38:45.663538+0000","last_became_peered":"2023-10-16T08:38:45.663538+0000","last_unstale":"2023-10-16T08:39:25.955495+0000","last_undegraded":"2023-10-16T08:39:25.955495+0000","last_fullsized":"2023-10-16T08:39:25.955495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402379+0000","last_change":"2023-10-16T08:38:44.646321+0000","last_active":"2023-10-16T08:39:26.402379+0000","last_peered":"2023-10-16T08:39:26.402379+0000","last_clean":"2023-10-16T08:39:26.402379+0000","last_became_active":"2023-10-16T08:38:44.645854+0000","last_became_peered":"2023-10-16T08:38:44.645854+0000","last_unstale":"2023-10-16T08:39:26.402379+0000","last_undegraded":"2023-10-16T08:39:26.402379+0000","last_fullsized":"2023-10-16T08:39:26.402379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:10.980982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6065,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":12130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":827392,"data_stored":674262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":1.3220000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.97699999999999998}]},{"osd":2,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":1.228}]},{"osd":3,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":1.363}]},{"osd":4,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":1.0569999999999999}]},{"osd":5,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.276},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":1.0900000000000001}]},{"osd":6,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":1.01}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":88579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.57499999999999996},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":0.63600000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":0.70599999999999996}]},{"osd":2,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.94099999999999995}]},{"osd":3,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":1.1930000000000001}]},{"osd":4,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":1.25}]},{"osd":5,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.3600000000000001}]},{"osd":7,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.04,"5min":2.04,"15min":2.04},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.3029999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":83402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.87}]},{"osd":1,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.165},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":1.0269999999999999}]},{"osd":2,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.133}]},{"osd":3,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.087},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.198}]},{"osd":4,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.98899999999999999}]},{"osd":6,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":0.84399999999999997}]},{"osd":7,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.111}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":83438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.004}]},{"osd":2,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.93799999999999994}]},{"osd":3,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.873}]},{"osd":5,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":1.05},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.754}]},{"osd":6,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":1.1060000000000001}]},{"osd":7,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.153,"5min":2.153,"15min":2.153},"last":1.079}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":88579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.51600000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.64400000000000002}]},{"osd":2,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.73099999999999998}]},{"osd":4,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.77400000000000002}]},{"osd":5,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.82699999999999996}]},{"osd":6,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":0.85999999999999999}]},{"osd":7,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.995}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":83312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.64700000000000002}]},{"osd":1,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.752}]},{"osd":3,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.79000000000000004}]},{"osd":4,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.119},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.93999999999999995}]},{"osd":5,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.23}]},{"osd":6,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.056},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":1.0840000000000001}]},{"osd":7,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.1970000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":83472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.48399999999999999}]},{"osd":2,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.68899999999999995}]},{"osd":3,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.60499999999999998}]},{"osd":4,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.70599999999999996}]},{"osd":5,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.123}]},{"osd":6,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.0049999999999999}]},{"osd":7,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.046},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.5449999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.94999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":835584,"data_stored":674318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.61499999999999999}]},{"osd":2,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.72199999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.56200000000000006}]},{"osd":4,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.75900000000000001}]},{"osd":5,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.97599999999999998}]},{"osd":6,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.91700000000000004}]},{"osd":7,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":1.0569999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:39:42.496 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:39:42.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.655+0000 7fda427b1640 1 Processor -- start 2023-10-16T08:39:42.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.655+0000 7fda427b1640 1 -- start start 2023-10-16T08:39:42.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.655+0000 7fda427b1640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fda3c0cd960 0x7fda3c0cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:42.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.655+0000 7fda427b1640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda3c0ce300 con 0x7fda3c15cce0 2023-10-16T08:39:42.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.655+0000 7fda427b1640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda3c0b7f80 con 0x7fda3c0d0d40 2023-10-16T08:39:42.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.655+0000 7fda427b1640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda3c0b7370 con 0x7fda3c0cd960 2023-10-16T08:39:42.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.655+0000 7fda3bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fda3c15cce0 0x7fda3c0b68d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45622/0 (socket says 172.21.15.40:45622) 2023-10-16T08:39:42.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.655+0000 7fda3bfff640 1 -- 172.21.15.40:0/3443255880 learned_addr learned my addr 172.21.15.40:0/3443255880 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:42.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.655+0000 7fda3b7fe640 1 --2- 172.21.15.40:0/3443255880 >> v2:172.21.15.40:3300/0 conn(0x7fda3c0cd960 0x7fda3c0cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:42.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.655+0000 7fda3affd640 1 -- 172.21.15.40:0/3443255880 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 306890795 0 0) 0x7fda3c0ce300 con 0x7fda3c15cce0 2023-10-16T08:39:42.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.655+0000 7fda3affd640 1 -- 172.21.15.40:0/3443255880 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda24003530 con 0x7fda3c15cce0 2023-10-16T08:39:42.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.655+0000 7fda3b7fe640 1 -- 172.21.15.40:0/3443255880 >> v1:172.21.15.86:6789/0 conn(0x7fda3c0d0d40 legacy=0x7fda3c0d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:42.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.655+0000 7fda3b7fe640 1 -- 172.21.15.40:0/3443255880 >> v1:172.21.15.40:6789/0 conn(0x7fda3c15cce0 legacy=0x7fda3c0b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:42.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.655+0000 7fda3b7fe640 1 -- 172.21.15.40:0/3443255880 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda3c0b7650 con 0x7fda3c0cd960 2023-10-16T08:39:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda3b7fe640 1 --2- 172.21.15.40:0/3443255880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c0cd960 0x7fda3c0cddc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fda300097c0 tx=0x7fda3001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5065fd497207b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda3affd640 1 -- 172.21.15.40:0/3443255880 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda300ab600 con 0x7fda3c0cd960 2023-10-16T08:39:42.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda3affd640 1 -- 172.21.15.40:0/3443255880 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda300a7050 con 0x7fda3c0cd960 2023-10-16T08:39:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda3affd640 1 -- 172.21.15.40:0/3443255880 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda300ab980 con 0x7fda3c0cd960 2023-10-16T08:39:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda427b1640 1 -- 172.21.15.40:0/3443255880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c0cd960 msgr2=0x7fda3c0cddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda427b1640 1 --2- 172.21.15.40:0/3443255880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c0cd960 0x7fda3c0cddc0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fda300097c0 tx=0x7fda3001c440 comp rx=0 tx=0).stop 2023-10-16T08:39:42.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda427b1640 1 -- 172.21.15.40:0/3443255880 shutdown_connections 2023-10-16T08:39:42.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda427b1640 1 --2- 172.21.15.40:0/3443255880 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c0cd960 0x7fda3c0cddc0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:42.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda427b1640 1 -- 172.21.15.40:0/3443255880 >> 172.21.15.40:0/3443255880 conn(0x7fda3c06a290 msgr2=0x7fda3c06a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:42.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda427b1640 1 -- 172.21.15.40:0/3443255880 shutdown_connections 2023-10-16T08:39:42.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda427b1640 1 -- 172.21.15.40:0/3443255880 wait complete. 2023-10-16T08:39:42.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda427b1640 1 Processor -- start 2023-10-16T08:39:42.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda427b1640 1 -- start start 2023-10-16T08:39:42.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda427b1640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda3c0cd960 0x7fda3c15fea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:42.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda427b1640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c0d0d40 0x7fda3c0c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:42.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda427b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3c15cce0 0x7fda3c14f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:42.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda427b1640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c0c6ca0 con 0x7fda3c0d0d40 2023-10-16T08:39:42.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda427b1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c0c6f80 con 0x7fda3c15cce0 2023-10-16T08:39:42.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda427b1640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda3c0c7260 con 0x7fda3c0cd960 2023-10-16T08:39:42.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda3b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c0d0d40 0x7fda3c0c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:42.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda3b7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c0d0d40 0x7fda3c0c7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50720/0 (socket says 172.21.15.40:50720) 2023-10-16T08:39:42.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda3b7fe640 1 -- 172.21.15.40:0/3497541663 learned_addr learned my addr 172.21.15.40:0/3497541663 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:42.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda40d4e640 1 --2- 172.21.15.40:0/3497541663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3c15cce0 0x7fda3c14f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:42.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda3b7fe640 1 -- 172.21.15.40:0/3497541663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda3c0cd960 msgr2=0x7fda3c15fea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:42.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda3b7fe640 1 --2- 172.21.15.40:0/3497541663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda3c0cd960 0x7fda3c15fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:42.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda3b7fe640 1 -- 172.21.15.40:0/3497541663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3c15cce0 msgr2=0x7fda3c14f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:42.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda3b7fe640 1 --2- 172.21.15.40:0/3497541663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3c15cce0 0x7fda3c14f150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:42.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda3b7fe640 1 -- 172.21.15.40:0/3497541663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda30009420 con 0x7fda3c0d0d40 2023-10-16T08:39:42.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda3b7fe640 1 --2- 172.21.15.40:0/3497541663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c0d0d40 0x7fda3c0c7d40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fda300ae040 tx=0x7fda300191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:42.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda38ff9640 1 -- 172.21.15.40:0/3497541663 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda300b4410 con 0x7fda3c0d0d40 2023-10-16T08:39:42.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda38ff9640 1 -- 172.21.15.40:0/3497541663 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda300b4570 con 0x7fda3c0d0d40 2023-10-16T08:39:42.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda38ff9640 1 -- 172.21.15.40:0/3497541663 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda300b4920 con 0x7fda3c0d0d40 2023-10-16T08:39:42.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda427b1640 1 -- 172.21.15.40:0/3497541663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda3c0c7540 con 0x7fda3c0d0d40 2023-10-16T08:39:42.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.659+0000 7fda427b1640 1 -- 172.21.15.40:0/3497541663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda3c1603e0 con 0x7fda3c0d0d40 2023-10-16T08:39:42.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.663+0000 7fda427b1640 1 -- 172.21.15.40:0/3497541663 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda3c04e9c0 con 0x7fda3c0d0d40 2023-10-16T08:39:42.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.663+0000 7fda38ff9640 1 -- 172.21.15.40:0/3497541663 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fda300b2040 con 0x7fda3c0d0d40 2023-10-16T08:39:42.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.663+0000 7fda38ff9640 1 --2- 172.21.15.40:0/3497541663 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fda280664d0 0x7fda28068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:42.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.663+0000 7fda38ff9640 1 -- 172.21.15.40:0/3497541663 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fda3011dca0 con 0x7fda3c0d0d40 2023-10-16T08:39:42.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.663+0000 7fda3bfff640 1 --2- 172.21.15.40:0/3497541663 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fda280664d0 0x7fda28068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.663+0000 7fda3bfff640 1 --2- 172.21.15.40:0/3497541663 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fda280664d0 0x7fda28068990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fda2c007fd0 tx=0x7fda2c007940 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:42.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.667+0000 7fda38ff9640 1 -- 172.21.15.40:0/3497541663 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fda300eb090 con 0x7fda3c0d0d40 2023-10-16T08:39:42.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.823+0000 7fda427b1640 1 -- 172.21.15.40:0/3497541663 --> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fda3c1605d0 con 0x7fda280664d0 2023-10-16T08:39:42.850 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.847+0000 7fda38ff9640 1 -- 172.21.15.40:0/3497541663 <== mgr.4111 v2:172.21.15.86:6832/1188970899 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286974 (secure 0 0 0) 0x7fda3c1605d0 con 0x7fda280664d0 2023-10-16T08:39:42.852 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:39:42.855 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-16T08:39:42.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.847+0000 7fda427b1640 1 -- 172.21.15.40:0/3497541663 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fda280664d0 msgr2=0x7fda28068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:42.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.847+0000 7fda427b1640 1 --2- 172.21.15.40:0/3497541663 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fda280664d0 0x7fda28068990 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fda2c007fd0 tx=0x7fda2c007940 comp rx=0 tx=0).stop 2023-10-16T08:39:42.856 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.851+0000 7fda427b1640 1 -- 172.21.15.40:0/3497541663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c0d0d40 msgr2=0x7fda3c0c7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:42.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.851+0000 7fda427b1640 1 --2- 172.21.15.40:0/3497541663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c0d0d40 0x7fda3c0c7d40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fda300ae040 tx=0x7fda300191e0 comp rx=0 tx=0).stop 2023-10-16T08:39:42.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.851+0000 7fda427b1640 1 -- 172.21.15.40:0/3497541663 shutdown_connections 2023-10-16T08:39:42.857 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.851+0000 7fda427b1640 1 --2- 172.21.15.40:0/3497541663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fda3c15cce0 0x7fda3c14f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:42.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.851+0000 7fda427b1640 1 --2- 172.21.15.40:0/3497541663 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fda280664d0 0x7fda28068990 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:42.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.851+0000 7fda427b1640 1 --2- 172.21.15.40:0/3497541663 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fda3c0d0d40 0x7fda3c0c7d40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:42.858 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.851+0000 7fda427b1640 1 --2- 172.21.15.40:0/3497541663 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fda3c0cd960 0x7fda3c15fea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:42.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.851+0000 7fda427b1640 1 -- 172.21.15.40:0/3497541663 >> 172.21.15.40:0/3497541663 conn(0x7fda3c06a290 msgr2=0x7fda3c152680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:42.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.851+0000 7fda427b1640 1 -- 172.21.15.40:0/3497541663 shutdown_connections 2023-10-16T08:39:42.859 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:42.851+0000 7fda427b1640 1 -- 172.21.15.40:0/3497541663 wait complete. 2023-10-16T08:39:42.873 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":49,"stamp":"2023-10-16T08:39:40.957890+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596433,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":218,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":169,"ondisk_log_size":169,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219200,"kb_used_data":3024,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755520,"statfs":{"total":773094113280,"available":772869652480,"internally_reserved":0,"allocated":3096576,"data_stored":1859362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004432"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.964601+0000","last_change":"2023-10-16T08:39:26.963087+0000","last_active":"2023-10-16T08:39:26.964601+0000","last_peered":"2023-10-16T08:39:26.964601+0000","last_clean":"2023-10-16T08:39:26.964601+0000","last_became_active":"2023-10-16T08:39:26.962446+0000","last_became_peered":"2023-10-16T08:39:26.962446+0000","last_unstale":"2023-10-16T08:39:26.964601+0000","last_undegraded":"2023-10-16T08:39:26.964601+0000","last_fullsized":"2023-10-16T08:39:26.964601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:33.190218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945446+0000","last_change":"2023-10-16T08:38:45.657659+0000","last_active":"2023-10-16T08:39:25.945446+0000","last_peered":"2023-10-16T08:39:25.945446+0000","last_clean":"2023-10-16T08:39:25.945446+0000","last_became_active":"2023-10-16T08:38:45.656884+0000","last_became_peered":"2023-10-16T08:38:45.656884+0000","last_unstale":"2023-10-16T08:39:25.945446+0000","last_undegraded":"2023-10-16T08:39:25.945446+0000","last_fullsized":"2023-10-16T08:39:25.945446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402116+0000","last_change":"2023-10-16T08:38:45.630125+0000","last_active":"2023-10-16T08:39:26.402116+0000","last_peered":"2023-10-16T08:39:26.402116+0000","last_clean":"2023-10-16T08:39:26.402116+0000","last_became_active":"2023-10-16T08:38:45.629297+0000","last_became_peered":"2023-10-16T08:38:45.629297+0000","last_unstale":"2023-10-16T08:39:26.402116+0000","last_undegraded":"2023-10-16T08:39:26.402116+0000","last_fullsized":"2023-10-16T08:39:26.402116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951941+0000","last_change":"2023-10-16T08:38:45.675996+0000","last_active":"2023-10-16T08:39:25.951941+0000","last_peered":"2023-10-16T08:39:25.951941+0000","last_clean":"2023-10-16T08:39:25.951941+0000","last_became_active":"2023-10-16T08:38:45.675327+0000","last_became_peered":"2023-10-16T08:38:45.675327+0000","last_unstale":"2023-10-16T08:39:25.951941+0000","last_undegraded":"2023-10-16T08:39:25.951941+0000","last_fullsized":"2023-10-16T08:39:25.951941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:33.190218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955358+0000","last_change":"2023-10-16T08:38:45.653550+0000","last_active":"2023-10-16T08:39:25.955358+0000","last_peered":"2023-10-16T08:39:25.955358+0000","last_clean":"2023-10-16T08:39:25.955358+0000","last_became_active":"2023-10-16T08:38:45.652808+0000","last_became_peered":"2023-10-16T08:38:45.652808+0000","last_unstale":"2023-10-16T08:39:25.955358+0000","last_undegraded":"2023-10-16T08:39:25.955358+0000","last_fullsized":"2023-10-16T08:39:25.955358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.950945+0000","last_change":"2023-10-16T08:38:45.675086+0000","last_active":"2023-10-16T08:39:25.950945+0000","last_peered":"2023-10-16T08:39:25.950945+0000","last_clean":"2023-10-16T08:39:25.950945+0000","last_became_active":"2023-10-16T08:38:45.671146+0000","last_became_peered":"2023-10-16T08:38:45.671146+0000","last_unstale":"2023-10-16T08:39:25.950945+0000","last_undegraded":"2023-10-16T08:39:25.950945+0000","last_fullsized":"2023-10-16T08:39:25.950945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954769+0000","last_change":"2023-10-16T08:38:45.652595+0000","last_active":"2023-10-16T08:39:25.954769+0000","last_peered":"2023-10-16T08:39:25.954769+0000","last_clean":"2023-10-16T08:39:25.954769+0000","last_became_active":"2023-10-16T08:38:45.651848+0000","last_became_peered":"2023-10-16T08:38:45.651848+0000","last_unstale":"2023-10-16T08:39:25.954769+0000","last_undegraded":"2023-10-16T08:39:25.954769+0000","last_fullsized":"2023-10-16T08:39:25.954769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:13.453321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955228+0000","last_change":"2023-10-16T08:38:45.668858+0000","last_active":"2023-10-16T08:39:25.955228+0000","last_peered":"2023-10-16T08:39:25.955228+0000","last_clean":"2023-10-16T08:39:25.955228+0000","last_became_active":"2023-10-16T08:38:45.664702+0000","last_became_peered":"2023-10-16T08:38:45.664702+0000","last_unstale":"2023-10-16T08:39:25.955228+0000","last_undegraded":"2023-10-16T08:39:25.955228+0000","last_fullsized":"2023-10-16T08:39:25.955228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955879+0000","last_change":"2023-10-16T08:38:45.667037+0000","last_active":"2023-10-16T08:39:25.955879+0000","last_peered":"2023-10-16T08:39:25.955879+0000","last_clean":"2023-10-16T08:39:25.955879+0000","last_became_active":"2023-10-16T08:38:45.666474+0000","last_became_peered":"2023-10-16T08:38:45.666474+0000","last_unstale":"2023-10-16T08:39:25.955879+0000","last_undegraded":"2023-10-16T08:39:25.955879+0000","last_fullsized":"2023-10-16T08:39:25.955879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:11:02.634006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949980+0000","last_change":"2023-10-16T08:38:45.669859+0000","last_active":"2023-10-16T08:39:25.949980+0000","last_peered":"2023-10-16T08:39:25.949980+0000","last_clean":"2023-10-16T08:39:25.949980+0000","last_became_active":"2023-10-16T08:38:45.668278+0000","last_became_peered":"2023-10-16T08:38:45.668278+0000","last_unstale":"2023-10-16T08:39:25.949980+0000","last_undegraded":"2023-10-16T08:39:25.949980+0000","last_fullsized":"2023-10-16T08:39:25.949980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949617+0000","last_change":"2023-10-16T08:38:45.664567+0000","last_active":"2023-10-16T08:39:25.949617+0000","last_peered":"2023-10-16T08:39:25.949617+0000","last_clean":"2023-10-16T08:39:25.949617+0000","last_became_active":"2023-10-16T08:38:45.663813+0000","last_became_peered":"2023-10-16T08:38:45.663813+0000","last_unstale":"2023-10-16T08:39:25.949617+0000","last_undegraded":"2023-10-16T08:39:25.949617+0000","last_fullsized":"2023-10-16T08:39:25.949617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403718+0000","last_change":"2023-10-16T08:38:45.675925+0000","last_active":"2023-10-16T08:39:26.403718+0000","last_peered":"2023-10-16T08:39:26.403718+0000","last_clean":"2023-10-16T08:39:26.403718+0000","last_became_active":"2023-10-16T08:38:45.675320+0000","last_became_peered":"2023-10-16T08:38:45.675320+0000","last_unstale":"2023-10-16T08:39:26.403718+0000","last_undegraded":"2023-10-16T08:39:26.403718+0000","last_fullsized":"2023-10-16T08:39:26.403718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:11.991837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956028+0000","last_change":"2023-10-16T08:38:45.668056+0000","last_active":"2023-10-16T08:39:25.956028+0000","last_peered":"2023-10-16T08:39:25.956028+0000","last_clean":"2023-10-16T08:39:25.956028+0000","last_became_active":"2023-10-16T08:38:45.667428+0000","last_became_peered":"2023-10-16T08:38:45.667428+0000","last_unstale":"2023-10-16T08:39:25.956028+0000","last_undegraded":"2023-10-16T08:39:25.956028+0000","last_fullsized":"2023-10-16T08:39:25.956028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.401718+0000","last_change":"2023-10-16T08:38:45.637518+0000","last_active":"2023-10-16T08:39:26.401718+0000","last_peered":"2023-10-16T08:39:26.401718+0000","last_clean":"2023-10-16T08:39:26.401718+0000","last_became_active":"2023-10-16T08:38:45.636785+0000","last_became_peered":"2023-10-16T08:38:45.636785+0000","last_unstale":"2023-10-16T08:39:26.401718+0000","last_undegraded":"2023-10-16T08:39:26.401718+0000","last_fullsized":"2023-10-16T08:39:26.401718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:11.870171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.944872+0000","last_change":"2023-10-16T08:38:45.656433+0000","last_active":"2023-10-16T08:39:25.944872+0000","last_peered":"2023-10-16T08:39:25.944872+0000","last_clean":"2023-10-16T08:39:25.944872+0000","last_became_active":"2023-10-16T08:38:45.655395+0000","last_became_peered":"2023-10-16T08:38:45.655395+0000","last_unstale":"2023-10-16T08:39:25.944872+0000","last_undegraded":"2023-10-16T08:39:25.944872+0000","last_fullsized":"2023-10-16T08:39:25.944872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.950540+0000","last_change":"2023-10-16T08:38:45.664192+0000","last_active":"2023-10-16T08:39:25.950540+0000","last_peered":"2023-10-16T08:39:25.950540+0000","last_clean":"2023-10-16T08:39:25.950540+0000","last_became_active":"2023-10-16T08:38:45.663312+0000","last_became_peered":"2023-10-16T08:38:45.663312+0000","last_unstale":"2023-10-16T08:39:25.950540+0000","last_undegraded":"2023-10-16T08:39:25.950540+0000","last_fullsized":"2023-10-16T08:39:25.950540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.016191+0000","last_change":"2023-10-16T08:39:27.013093+0000","last_active":"2023-10-16T08:39:27.016191+0000","last_peered":"2023-10-16T08:39:27.016191+0000","last_clean":"2023-10-16T08:39:27.016191+0000","last_became_active":"2023-10-16T08:39:27.012068+0000","last_became_peered":"2023-10-16T08:39:27.012068+0000","last_unstale":"2023-10-16T08:39:27.016191+0000","last_undegraded":"2023-10-16T08:39:27.016191+0000","last_fullsized":"2023-10-16T08:39:27.016191+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945605+0000","last_change":"2023-10-16T08:38:45.631488+0000","last_active":"2023-10-16T08:39:25.945605+0000","last_peered":"2023-10-16T08:39:25.945605+0000","last_clean":"2023-10-16T08:39:25.945605+0000","last_became_active":"2023-10-16T08:38:45.630641+0000","last_became_peered":"2023-10-16T08:38:45.630641+0000","last_unstale":"2023-10-16T08:39:25.945605+0000","last_undegraded":"2023-10-16T08:39:25.945605+0000","last_fullsized":"2023-10-16T08:39:25.945605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:11.991837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955988+0000","last_change":"2023-10-16T08:38:45.650087+0000","last_active":"2023-10-16T08:39:25.955988+0000","last_peered":"2023-10-16T08:39:25.955988+0000","last_clean":"2023-10-16T08:39:25.955988+0000","last_became_active":"2023-10-16T08:38:45.649475+0000","last_became_peered":"2023-10-16T08:38:45.649475+0000","last_unstale":"2023-10-16T08:39:25.955988+0000","last_undegraded":"2023-10-16T08:39:25.955988+0000","last_fullsized":"2023-10-16T08:39:25.955988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:26.292137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949532+0000","last_change":"2023-10-16T08:38:45.667336+0000","last_active":"2023-10-16T08:39:25.949532+0000","last_peered":"2023-10-16T08:39:25.949532+0000","last_clean":"2023-10-16T08:39:25.949532+0000","last_became_active":"2023-10-16T08:38:45.665291+0000","last_became_peered":"2023-10-16T08:38:45.665291+0000","last_unstale":"2023-10-16T08:39:25.949532+0000","last_undegraded":"2023-10-16T08:39:25.949532+0000","last_fullsized":"2023-10-16T08:39:25.949532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402191+0000","last_change":"2023-10-16T08:38:45.654036+0000","last_active":"2023-10-16T08:39:26.402191+0000","last_peered":"2023-10-16T08:39:26.402191+0000","last_clean":"2023-10-16T08:39:26.402191+0000","last_became_active":"2023-10-16T08:38:45.652412+0000","last_became_peered":"2023-10-16T08:38:45.652412+0000","last_unstale":"2023-10-16T08:39:26.402191+0000","last_undegraded":"2023-10-16T08:39:26.402191+0000","last_fullsized":"2023-10-16T08:39:26.402191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951156+0000","last_change":"2023-10-16T08:38:45.677644+0000","last_active":"2023-10-16T08:39:25.951156+0000","last_peered":"2023-10-16T08:39:25.951156+0000","last_clean":"2023-10-16T08:39:25.951156+0000","last_became_active":"2023-10-16T08:38:45.676844+0000","last_became_peered":"2023-10-16T08:38:45.676844+0000","last_unstale":"2023-10-16T08:39:25.951156+0000","last_undegraded":"2023-10-16T08:39:25.951156+0000","last_fullsized":"2023-10-16T08:39:25.951156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955256+0000","last_change":"2023-10-16T08:38:45.663771+0000","last_active":"2023-10-16T08:39:25.955256+0000","last_peered":"2023-10-16T08:39:25.955256+0000","last_clean":"2023-10-16T08:39:25.955256+0000","last_became_active":"2023-10-16T08:38:45.662944+0000","last_became_peered":"2023-10-16T08:38:45.662944+0000","last_unstale":"2023-10-16T08:39:25.955256+0000","last_undegraded":"2023-10-16T08:39:25.955256+0000","last_fullsized":"2023-10-16T08:39:25.955256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.950669+0000","last_change":"2023-10-16T08:38:45.667398+0000","last_active":"2023-10-16T08:39:25.950669+0000","last_peered":"2023-10-16T08:39:25.950669+0000","last_clean":"2023-10-16T08:39:25.950669+0000","last_became_active":"2023-10-16T08:38:45.666641+0000","last_became_peered":"2023-10-16T08:38:45.666641+0000","last_unstale":"2023-10-16T08:39:25.950669+0000","last_undegraded":"2023-10-16T08:39:25.950669+0000","last_fullsized":"2023-10-16T08:39:25.950669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955685+0000","last_change":"2023-10-16T08:38:45.670526+0000","last_active":"2023-10-16T08:39:25.955685+0000","last_peered":"2023-10-16T08:39:25.955685+0000","last_clean":"2023-10-16T08:39:25.955685+0000","last_became_active":"2023-10-16T08:38:45.669734+0000","last_became_peered":"2023-10-16T08:38:45.669734+0000","last_unstale":"2023-10-16T08:39:25.955685+0000","last_undegraded":"2023-10-16T08:39:25.955685+0000","last_fullsized":"2023-10-16T08:39:25.955685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949999+0000","last_change":"2023-10-16T08:38:45.670429+0000","last_active":"2023-10-16T08:39:25.949999+0000","last_peered":"2023-10-16T08:39:25.949999+0000","last_clean":"2023-10-16T08:39:25.949999+0000","last_became_active":"2023-10-16T08:38:45.665972+0000","last_became_peered":"2023-10-16T08:38:45.665972+0000","last_unstale":"2023-10-16T08:39:25.949999+0000","last_undegraded":"2023-10-16T08:39:25.949999+0000","last_fullsized":"2023-10-16T08:39:25.949999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945074+0000","last_change":"2023-10-16T08:38:45.667047+0000","last_active":"2023-10-16T08:39:25.945074+0000","last_peered":"2023-10-16T08:39:25.945074+0000","last_clean":"2023-10-16T08:39:25.945074+0000","last_became_active":"2023-10-16T08:38:45.666403+0000","last_became_peered":"2023-10-16T08:38:45.666403+0000","last_unstale":"2023-10-16T08:39:25.945074+0000","last_undegraded":"2023-10-16T08:39:25.945074+0000","last_fullsized":"2023-10-16T08:39:25.945074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956047+0000","last_change":"2023-10-16T08:38:45.668007+0000","last_active":"2023-10-16T08:39:25.956047+0000","last_peered":"2023-10-16T08:39:25.956047+0000","last_clean":"2023-10-16T08:39:25.956047+0000","last_became_active":"2023-10-16T08:38:45.664572+0000","last_became_peered":"2023-10-16T08:38:45.664572+0000","last_unstale":"2023-10-16T08:39:25.956047+0000","last_undegraded":"2023-10-16T08:39:25.956047+0000","last_fullsized":"2023-10-16T08:39:25.956047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402369+0000","last_change":"2023-10-16T08:38:45.667378+0000","last_active":"2023-10-16T08:39:26.402369+0000","last_peered":"2023-10-16T08:39:26.402369+0000","last_clean":"2023-10-16T08:39:26.402369+0000","last_became_active":"2023-10-16T08:38:45.666547+0000","last_became_peered":"2023-10-16T08:38:45.666547+0000","last_unstale":"2023-10-16T08:39:26.402369+0000","last_undegraded":"2023-10-16T08:39:26.402369+0000","last_fullsized":"2023-10-16T08:39:26.402369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.013174+0000","last_change":"2023-10-16T08:39:27.010902+0000","last_active":"2023-10-16T08:39:27.013174+0000","last_peered":"2023-10-16T08:39:27.013174+0000","last_clean":"2023-10-16T08:39:27.013174+0000","last_became_active":"2023-10-16T08:39:27.009983+0000","last_became_peered":"2023-10-16T08:39:27.009983+0000","last_unstale":"2023-10-16T08:39:27.013174+0000","last_undegraded":"2023-10-16T08:39:27.013174+0000","last_fullsized":"2023-10-16T08:39:27.013174+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.014469+0000","last_change":"2023-10-16T08:39:27.014469+0000","last_active":"2023-10-16T08:39:27.014469+0000","last_peered":"2023-10-16T08:39:27.014469+0000","last_clean":"2023-10-16T08:39:27.014469+0000","last_became_active":"2023-10-16T08:39:27.013658+0000","last_became_peered":"2023-10-16T08:39:27.013658+0000","last_unstale":"2023-10-16T08:39:27.014469+0000","last_undegraded":"2023-10-16T08:39:27.014469+0000","last_fullsized":"2023-10-16T08:39:27.014469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949746+0000","last_change":"2023-10-16T08:38:45.638183+0000","last_active":"2023-10-16T08:39:25.949746+0000","last_peered":"2023-10-16T08:39:25.949746+0000","last_clean":"2023-10-16T08:39:25.949746+0000","last_became_active":"2023-10-16T08:38:45.636762+0000","last_became_peered":"2023-10-16T08:38:45.636762+0000","last_unstale":"2023-10-16T08:39:25.949746+0000","last_undegraded":"2023-10-16T08:39:25.949746+0000","last_fullsized":"2023-10-16T08:39:25.949746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403351+0000","last_change":"2023-10-16T08:38:45.659464+0000","last_active":"2023-10-16T08:39:26.403351+0000","last_peered":"2023-10-16T08:39:26.403351+0000","last_clean":"2023-10-16T08:39:26.403351+0000","last_became_active":"2023-10-16T08:38:45.658808+0000","last_became_peered":"2023-10-16T08:38:45.658808+0000","last_unstale":"2023-10-16T08:39:26.403351+0000","last_undegraded":"2023-10-16T08:39:26.403351+0000","last_fullsized":"2023-10-16T08:39:26.403351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:25.386221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956227+0000","last_change":"2023-10-16T08:38:44.644122+0000","last_active":"2023-10-16T08:39:25.956227+0000","last_peered":"2023-10-16T08:39:25.956227+0000","last_clean":"2023-10-16T08:39:25.956227+0000","last_became_active":"2023-10-16T08:38:44.643614+0000","last_became_peered":"2023-10-16T08:38:44.643614+0000","last_unstale":"2023-10-16T08:39:25.956227+0000","last_undegraded":"2023-10-16T08:39:25.956227+0000","last_fullsized":"2023-10-16T08:39:25.956227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402402+0000","last_change":"2023-10-16T08:38:45.659854+0000","last_active":"2023-10-16T08:39:26.402402+0000","last_peered":"2023-10-16T08:39:26.402402+0000","last_clean":"2023-10-16T08:39:26.402402+0000","last_became_active":"2023-10-16T08:38:45.658409+0000","last_became_peered":"2023-10-16T08:38:45.658409+0000","last_unstale":"2023-10-16T08:39:26.402402+0000","last_undegraded":"2023-10-16T08:39:26.402402+0000","last_fullsized":"2023-10-16T08:39:26.402402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949532+0000","last_change":"2023-10-16T08:38:44.658819+0000","last_active":"2023-10-16T08:39:25.949532+0000","last_peered":"2023-10-16T08:39:25.949532+0000","last_clean":"2023-10-16T08:39:25.949532+0000","last_became_active":"2023-10-16T08:38:44.658257+0000","last_became_peered":"2023-10-16T08:38:44.658257+0000","last_unstale":"2023-10-16T08:39:25.949532+0000","last_undegraded":"2023-10-16T08:39:25.949532+0000","last_fullsized":"2023-10-16T08:39:25.949532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:54.848445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955337+0000","last_change":"2023-10-16T08:38:44.647536+0000","last_active":"2023-10-16T08:39:25.955337+0000","last_peered":"2023-10-16T08:39:25.955337+0000","last_clean":"2023-10-16T08:39:25.955337+0000","last_became_active":"2023-10-16T08:38:44.645931+0000","last_became_peered":"2023-10-16T08:38:44.645931+0000","last_unstale":"2023-10-16T08:39:25.955337+0000","last_undegraded":"2023-10-16T08:39:25.955337+0000","last_fullsized":"2023-10-16T08:39:25.955337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955491+0000","last_change":"2023-10-16T08:38:45.662855+0000","last_active":"2023-10-16T08:39:25.955491+0000","last_peered":"2023-10-16T08:39:25.955491+0000","last_clean":"2023-10-16T08:39:25.955491+0000","last_became_active":"2023-10-16T08:38:45.662072+0000","last_became_peered":"2023-10-16T08:38:45.662072+0000","last_unstale":"2023-10-16T08:39:25.955491+0000","last_undegraded":"2023-10-16T08:39:25.955491+0000","last_fullsized":"2023-10-16T08:39:25.955491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.950636+0000","last_change":"2023-10-16T08:38:44.657651+0000","last_active":"2023-10-16T08:39:25.950636+0000","last_peered":"2023-10-16T08:39:25.950636+0000","last_clean":"2023-10-16T08:39:25.950636+0000","last_became_active":"2023-10-16T08:38:44.657065+0000","last_became_peered":"2023-10-16T08:38:44.657065+0000","last_unstale":"2023-10-16T08:39:25.950636+0000","last_undegraded":"2023-10-16T08:39:25.950636+0000","last_fullsized":"2023-10-16T08:39:25.950636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.944769+0000","last_change":"2023-10-16T08:38:45.668048+0000","last_active":"2023-10-16T08:39:25.944769+0000","last_peered":"2023-10-16T08:39:25.944769+0000","last_clean":"2023-10-16T08:39:25.944769+0000","last_became_active":"2023-10-16T08:38:45.667413+0000","last_became_peered":"2023-10-16T08:38:45.667413+0000","last_unstale":"2023-10-16T08:39:25.944769+0000","last_undegraded":"2023-10-16T08:39:25.944769+0000","last_fullsized":"2023-10-16T08:39:25.944769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949842+0000","last_change":"2023-10-16T08:38:44.663188+0000","last_active":"2023-10-16T08:39:25.949842+0000","last_peered":"2023-10-16T08:39:25.949842+0000","last_clean":"2023-10-16T08:39:25.949842+0000","last_became_active":"2023-10-16T08:38:44.662383+0000","last_became_peered":"2023-10-16T08:38:44.662383+0000","last_unstale":"2023-10-16T08:39:25.949842+0000","last_undegraded":"2023-10-16T08:39:25.949842+0000","last_fullsized":"2023-10-16T08:39:25.949842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:24.375366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.953576+0000","last_change":"2023-10-16T08:38:45.679271+0000","last_active":"2023-10-16T08:39:25.953576+0000","last_peered":"2023-10-16T08:39:25.953576+0000","last_clean":"2023-10-16T08:39:25.953576+0000","last_became_active":"2023-10-16T08:38:45.678050+0000","last_became_peered":"2023-10-16T08:38:45.678050+0000","last_unstale":"2023-10-16T08:39:25.953576+0000","last_undegraded":"2023-10-16T08:39:25.953576+0000","last_fullsized":"2023-10-16T08:39:25.953576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.953752+0000","last_change":"2023-10-16T08:38:44.653478+0000","last_active":"2023-10-16T08:39:25.953752+0000","last_peered":"2023-10-16T08:39:25.953752+0000","last_clean":"2023-10-16T08:39:25.953752+0000","last_became_active":"2023-10-16T08:38:44.652618+0000","last_became_peered":"2023-10-16T08:38:44.652618+0000","last_unstale":"2023-10-16T08:39:25.953752+0000","last_undegraded":"2023-10-16T08:39:25.953752+0000","last_fullsized":"2023-10-16T08:39:25.953752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402370+0000","last_change":"2023-10-16T08:38:45.656603+0000","last_active":"2023-10-16T08:39:26.402370+0000","last_peered":"2023-10-16T08:39:26.402370+0000","last_clean":"2023-10-16T08:39:26.402370+0000","last_became_active":"2023-10-16T08:38:45.655213+0000","last_became_peered":"2023-10-16T08:38:45.655213+0000","last_unstale":"2023-10-16T08:39:26.402370+0000","last_undegraded":"2023-10-16T08:39:26.402370+0000","last_fullsized":"2023-10-16T08:39:26.402370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945084+0000","last_change":"2023-10-16T08:38:44.651637+0000","last_active":"2023-10-16T08:39:25.945084+0000","last_peered":"2023-10-16T08:39:25.945084+0000","last_clean":"2023-10-16T08:39:25.945084+0000","last_became_active":"2023-10-16T08:38:44.651359+0000","last_became_peered":"2023-10-16T08:38:44.651359+0000","last_unstale":"2023-10-16T08:39:25.945084+0000","last_undegraded":"2023-10-16T08:39:25.945084+0000","last_fullsized":"2023-10-16T08:39:25.945084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:10.859316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403856+0000","last_change":"2023-10-16T08:38:45.661741+0000","last_active":"2023-10-16T08:39:26.403856+0000","last_peered":"2023-10-16T08:39:26.403856+0000","last_clean":"2023-10-16T08:39:26.403856+0000","last_became_active":"2023-10-16T08:38:45.660611+0000","last_became_peered":"2023-10-16T08:38:45.660611+0000","last_unstale":"2023-10-16T08:39:26.403856+0000","last_undegraded":"2023-10-16T08:39:26.403856+0000","last_fullsized":"2023-10-16T08:39:26.403856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:55.859300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'18","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:35.375472+0000","last_change":"2023-10-16T08:38:44.647593+0000","last_active":"2023-10-16T08:39:35.375472+0000","last_peered":"2023-10-16T08:39:35.375472+0000","last_clean":"2023-10-16T08:39:35.375472+0000","last_became_active":"2023-10-16T08:38:44.647001+0000","last_became_peered":"2023-10-16T08:38:44.647001+0000","last_unstale":"2023-10-16T08:39:35.375472+0000","last_undegraded":"2023-10-16T08:39:35.375472+0000","last_fullsized":"2023-10-16T08:39:35.375472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5267,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955477+0000","last_change":"2023-10-16T08:38:45.668629+0000","last_active":"2023-10-16T08:39:25.955477+0000","last_peered":"2023-10-16T08:39:25.955477+0000","last_clean":"2023-10-16T08:39:25.955477+0000","last_became_active":"2023-10-16T08:38:45.667334+0000","last_became_peered":"2023-10-16T08:38:45.667334+0000","last_unstale":"2023-10-16T08:39:25.955477+0000","last_undegraded":"2023-10-16T08:39:25.955477+0000","last_fullsized":"2023-10-16T08:39:25.955477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402610+0000","last_change":"2023-10-16T08:38:44.655832+0000","last_active":"2023-10-16T08:39:26.402610+0000","last_peered":"2023-10-16T08:39:26.402610+0000","last_clean":"2023-10-16T08:39:26.402610+0000","last_became_active":"2023-10-16T08:38:44.655469+0000","last_became_peered":"2023-10-16T08:38:44.655469+0000","last_unstale":"2023-10-16T08:39:26.402610+0000","last_undegraded":"2023-10-16T08:39:26.402610+0000","last_fullsized":"2023-10-16T08:39:26.402610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:19.768762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.401855+0000","last_change":"2023-10-16T08:38:45.653688+0000","last_active":"2023-10-16T08:39:26.401855+0000","last_peered":"2023-10-16T08:39:26.401855+0000","last_clean":"2023-10-16T08:39:26.401855+0000","last_became_active":"2023-10-16T08:38:45.650563+0000","last_became_peered":"2023-10-16T08:38:45.650563+0000","last_unstale":"2023-10-16T08:39:26.401855+0000","last_undegraded":"2023-10-16T08:39:26.401855+0000","last_fullsized":"2023-10-16T08:39:26.401855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:29.539285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951818+0000","last_change":"2023-10-16T08:38:44.662816+0000","last_active":"2023-10-16T08:39:25.951818+0000","last_peered":"2023-10-16T08:39:25.951818+0000","last_clean":"2023-10-16T08:39:25.951818+0000","last_became_active":"2023-10-16T08:38:44.662110+0000","last_became_peered":"2023-10-16T08:38:44.662110+0000","last_unstale":"2023-10-16T08:39:25.951818+0000","last_undegraded":"2023-10-16T08:39:25.951818+0000","last_fullsized":"2023-10-16T08:39:25.951818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:00.201724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955811+0000","last_change":"2023-10-16T08:38:45.662166+0000","last_active":"2023-10-16T08:39:25.955811+0000","last_peered":"2023-10-16T08:39:25.955811+0000","last_clean":"2023-10-16T08:39:25.955811+0000","last_became_active":"2023-10-16T08:38:45.661595+0000","last_became_peered":"2023-10-16T08:38:45.661595+0000","last_unstale":"2023-10-16T08:39:25.955811+0000","last_undegraded":"2023-10-16T08:39:25.955811+0000","last_fullsized":"2023-10-16T08:39:25.955811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:02:39.840175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949503+0000","last_change":"2023-10-16T08:38:44.657766+0000","last_active":"2023-10-16T08:39:25.949503+0000","last_peered":"2023-10-16T08:39:25.949503+0000","last_clean":"2023-10-16T08:39:25.949503+0000","last_became_active":"2023-10-16T08:38:44.657153+0000","last_became_peered":"2023-10-16T08:38:44.657153+0000","last_unstale":"2023-10-16T08:39:25.949503+0000","last_undegraded":"2023-10-16T08:39:25.949503+0000","last_fullsized":"2023-10-16T08:39:25.949503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949501+0000","last_change":"2023-10-16T08:38:45.637591+0000","last_active":"2023-10-16T08:39:25.949501+0000","last_peered":"2023-10-16T08:39:25.949501+0000","last_clean":"2023-10-16T08:39:25.949501+0000","last_became_active":"2023-10-16T08:38:45.635633+0000","last_became_peered":"2023-10-16T08:38:45.635633+0000","last_unstale":"2023-10-16T08:39:25.949501+0000","last_undegraded":"2023-10-16T08:39:25.949501+0000","last_fullsized":"2023-10-16T08:39:25.949501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.946160+0000","last_change":"2023-10-16T08:38:44.638297+0000","last_active":"2023-10-16T08:39:25.946160+0000","last_peered":"2023-10-16T08:39:25.946160+0000","last_clean":"2023-10-16T08:39:25.946160+0000","last_became_active":"2023-10-16T08:38:44.637640+0000","last_became_peered":"2023-10-16T08:38:44.637640+0000","last_unstale":"2023-10-16T08:39:25.946160+0000","last_undegraded":"2023-10-16T08:39:25.946160+0000","last_fullsized":"2023-10-16T08:39:25.946160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:16.003943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954941+0000","last_change":"2023-10-16T08:38:45.657286+0000","last_active":"2023-10-16T08:39:25.954941+0000","last_peered":"2023-10-16T08:39:25.954941+0000","last_clean":"2023-10-16T08:39:25.954941+0000","last_became_active":"2023-10-16T08:38:45.656651+0000","last_became_peered":"2023-10-16T08:38:45.656651+0000","last_unstale":"2023-10-16T08:39:25.954941+0000","last_undegraded":"2023-10-16T08:39:25.954941+0000","last_fullsized":"2023-10-16T08:39:25.954941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:28.052483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403973+0000","last_change":"2023-10-16T08:38:44.648434+0000","last_active":"2023-10-16T08:39:26.403973+0000","last_peered":"2023-10-16T08:39:26.403973+0000","last_clean":"2023-10-16T08:39:26.403973+0000","last_became_active":"2023-10-16T08:38:44.647887+0000","last_became_peered":"2023-10-16T08:38:44.647887+0000","last_unstale":"2023-10-16T08:39:26.403973+0000","last_undegraded":"2023-10-16T08:39:26.403973+0000","last_fullsized":"2023-10-16T08:39:26.403973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954988+0000","last_change":"2023-10-16T08:38:45.661965+0000","last_active":"2023-10-16T08:39:25.954988+0000","last_peered":"2023-10-16T08:39:25.954988+0000","last_clean":"2023-10-16T08:39:25.954988+0000","last_became_active":"2023-10-16T08:38:45.656750+0000","last_became_peered":"2023-10-16T08:38:45.656750+0000","last_unstale":"2023-10-16T08:39:25.954988+0000","last_undegraded":"2023-10-16T08:39:25.954988+0000","last_fullsized":"2023-10-16T08:39:25.954988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945809+0000","last_change":"2023-10-16T08:38:44.647524+0000","last_active":"2023-10-16T08:39:25.945809+0000","last_peered":"2023-10-16T08:39:25.945809+0000","last_clean":"2023-10-16T08:39:25.945809+0000","last_became_active":"2023-10-16T08:38:44.646042+0000","last_became_peered":"2023-10-16T08:38:44.646042+0000","last_unstale":"2023-10-16T08:39:25.945809+0000","last_undegraded":"2023-10-16T08:39:25.945809+0000","last_fullsized":"2023-10-16T08:39:25.945809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:24.375366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955916+0000","last_change":"2023-10-16T08:38:45.670969+0000","last_active":"2023-10-16T08:39:25.955916+0000","last_peered":"2023-10-16T08:39:25.955916+0000","last_clean":"2023-10-16T08:39:25.955916+0000","last_became_active":"2023-10-16T08:38:45.670415+0000","last_became_peered":"2023-10-16T08:38:45.670415+0000","last_unstale":"2023-10-16T08:39:25.955916+0000","last_undegraded":"2023-10-16T08:39:25.955916+0000","last_fullsized":"2023-10-16T08:39:25.955916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:15:00.472153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945149+0000","last_change":"2023-10-16T08:38:44.655744+0000","last_active":"2023-10-16T08:39:25.945149+0000","last_peered":"2023-10-16T08:39:25.945149+0000","last_clean":"2023-10-16T08:39:25.945149+0000","last_became_active":"2023-10-16T08:38:44.655260+0000","last_became_peered":"2023-10-16T08:38:44.655260+0000","last_unstale":"2023-10-16T08:39:25.945149+0000","last_undegraded":"2023-10-16T08:39:25.945149+0000","last_fullsized":"2023-10-16T08:39:25.945149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:54.848445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951801+0000","last_change":"2023-10-16T08:38:45.668230+0000","last_active":"2023-10-16T08:39:25.951801+0000","last_peered":"2023-10-16T08:39:25.951801+0000","last_clean":"2023-10-16T08:39:25.951801+0000","last_became_active":"2023-10-16T08:38:45.667627+0000","last_became_peered":"2023-10-16T08:38:45.667627+0000","last_unstale":"2023-10-16T08:39:25.951801+0000","last_undegraded":"2023-10-16T08:39:25.951801+0000","last_fullsized":"2023-10-16T08:39:25.951801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:28.052483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945644+0000","last_change":"2023-10-16T08:38:44.662118+0000","last_active":"2023-10-16T08:39:25.945644+0000","last_peered":"2023-10-16T08:39:25.945644+0000","last_clean":"2023-10-16T08:39:25.945644+0000","last_became_active":"2023-10-16T08:38:44.661736+0000","last_became_peered":"2023-10-16T08:38:44.661736+0000","last_unstale":"2023-10-16T08:39:25.945644+0000","last_undegraded":"2023-10-16T08:39:25.945644+0000","last_fullsized":"2023-10-16T08:39:25.945644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:49:40.967349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955744+0000","last_change":"2023-10-16T08:38:45.667423+0000","last_active":"2023-10-16T08:39:25.955744+0000","last_peered":"2023-10-16T08:39:25.955744+0000","last_clean":"2023-10-16T08:39:25.955744+0000","last_became_active":"2023-10-16T08:38:45.666390+0000","last_became_peered":"2023-10-16T08:38:45.666390+0000","last_unstale":"2023-10-16T08:39:25.955744+0000","last_undegraded":"2023-10-16T08:39:25.955744+0000","last_fullsized":"2023-10-16T08:39:25.955744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:33.190218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":167,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954612+0000","last_change":"2023-10-16T08:38:34.313511+0000","last_active":"2023-10-16T08:39:25.954612+0000","last_peered":"2023-10-16T08:39:25.954612+0000","last_clean":"2023-10-16T08:39:25.954612+0000","last_became_active":"2023-10-16T08:38:34.310944+0000","last_became_peered":"2023-10-16T08:38:34.310944+0000","last_unstale":"2023-10-16T08:39:25.954612+0000","last_undegraded":"2023-10-16T08:39:25.954612+0000","last_fullsized":"2023-10-16T08:39:25.954612+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:33.264692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:33.264692+0000","last_clean_scrub_stamp":"2023-10-16T08:38:33.264692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:50.170461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951158+0000","last_change":"2023-10-16T08:38:44.655223+0000","last_active":"2023-10-16T08:39:25.951158+0000","last_peered":"2023-10-16T08:39:25.951158+0000","last_clean":"2023-10-16T08:39:25.951158+0000","last_became_active":"2023-10-16T08:38:44.654277+0000","last_became_peered":"2023-10-16T08:38:44.654277+0000","last_unstale":"2023-10-16T08:39:25.951158+0000","last_undegraded":"2023-10-16T08:39:25.951158+0000","last_fullsized":"2023-10-16T08:39:25.951158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:03.462433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955641+0000","last_change":"2023-10-16T08:38:45.679868+0000","last_active":"2023-10-16T08:39:25.955641+0000","last_peered":"2023-10-16T08:39:25.955641+0000","last_clean":"2023-10-16T08:39:25.955641+0000","last_became_active":"2023-10-16T08:38:45.679409+0000","last_became_peered":"2023-10-16T08:38:45.679409+0000","last_unstale":"2023-10-16T08:39:25.955641+0000","last_undegraded":"2023-10-16T08:39:25.955641+0000","last_fullsized":"2023-10-16T08:39:25.955641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954528+0000","last_change":"2023-10-16T08:38:44.652152+0000","last_active":"2023-10-16T08:39:25.954528+0000","last_peered":"2023-10-16T08:39:25.954528+0000","last_clean":"2023-10-16T08:39:25.954528+0000","last_became_active":"2023-10-16T08:38:44.651534+0000","last_became_peered":"2023-10-16T08:38:44.651534+0000","last_unstale":"2023-10-16T08:39:25.954528+0000","last_undegraded":"2023-10-16T08:39:25.954528+0000","last_fullsized":"2023-10-16T08:39:25.954528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.401719+0000","last_change":"2023-10-16T08:38:45.651786+0000","last_active":"2023-10-16T08:39:26.401719+0000","last_peered":"2023-10-16T08:39:26.401719+0000","last_clean":"2023-10-16T08:39:26.401719+0000","last_became_active":"2023-10-16T08:38:45.650715+0000","last_became_peered":"2023-10-16T08:38:45.650715+0000","last_unstale":"2023-10-16T08:39:26.401719+0000","last_undegraded":"2023-10-16T08:39:26.401719+0000","last_fullsized":"2023-10-16T08:39:26.401719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:29.799799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.946109+0000","last_change":"2023-10-16T08:38:44.645933+0000","last_active":"2023-10-16T08:39:25.946109+0000","last_peered":"2023-10-16T08:39:25.946109+0000","last_clean":"2023-10-16T08:39:25.946109+0000","last_became_active":"2023-10-16T08:38:44.645451+0000","last_became_peered":"2023-10-16T08:38:44.645451+0000","last_unstale":"2023-10-16T08:39:25.946109+0000","last_undegraded":"2023-10-16T08:39:25.946109+0000","last_fullsized":"2023-10-16T08:39:25.946109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.944780+0000","last_change":"2023-10-16T08:38:45.641499+0000","last_active":"2023-10-16T08:39:25.944780+0000","last_peered":"2023-10-16T08:39:25.944780+0000","last_clean":"2023-10-16T08:39:25.944780+0000","last_became_active":"2023-10-16T08:38:45.640765+0000","last_became_peered":"2023-10-16T08:38:45.640765+0000","last_unstale":"2023-10-16T08:39:25.944780+0000","last_undegraded":"2023-10-16T08:39:25.944780+0000","last_fullsized":"2023-10-16T08:39:25.944780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.952050+0000","last_change":"2023-10-16T08:38:44.655401+0000","last_active":"2023-10-16T08:39:25.952050+0000","last_peered":"2023-10-16T08:39:25.952050+0000","last_clean":"2023-10-16T08:39:25.952050+0000","last_became_active":"2023-10-16T08:38:44.654622+0000","last_became_peered":"2023-10-16T08:38:44.654622+0000","last_unstale":"2023-10-16T08:39:25.952050+0000","last_undegraded":"2023-10-16T08:39:25.952050+0000","last_fullsized":"2023-10-16T08:39:25.952050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:48.705915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403580+0000","last_change":"2023-10-16T08:38:45.659377+0000","last_active":"2023-10-16T08:39:26.403580+0000","last_peered":"2023-10-16T08:39:26.403580+0000","last_clean":"2023-10-16T08:39:26.403580+0000","last_became_active":"2023-10-16T08:38:45.658594+0000","last_became_peered":"2023-10-16T08:38:45.658594+0000","last_unstale":"2023-10-16T08:39:26.403580+0000","last_undegraded":"2023-10-16T08:39:26.403580+0000","last_fullsized":"2023-10-16T08:39:26.403580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:56.738058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945087+0000","last_change":"2023-10-16T08:38:44.634158+0000","last_active":"2023-10-16T08:39:25.945087+0000","last_peered":"2023-10-16T08:39:25.945087+0000","last_clean":"2023-10-16T08:39:25.945087+0000","last_became_active":"2023-10-16T08:38:44.633854+0000","last_became_peered":"2023-10-16T08:38:44.633854+0000","last_unstale":"2023-10-16T08:39:25.945087+0000","last_undegraded":"2023-10-16T08:39:25.945087+0000","last_fullsized":"2023-10-16T08:39:25.945087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:55.727203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.015974+0000","last_change":"2023-10-16T08:39:27.015974+0000","last_active":"2023-10-16T08:39:27.015974+0000","last_peered":"2023-10-16T08:39:27.015974+0000","last_clean":"2023-10-16T08:39:27.015974+0000","last_became_active":"2023-10-16T08:39:27.015266+0000","last_became_peered":"2023-10-16T08:39:27.015266+0000","last_unstale":"2023-10-16T08:39:27.015974+0000","last_undegraded":"2023-10-16T08:39:27.015974+0000","last_fullsized":"2023-10-16T08:39:27.015974+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:41:52.951028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956207+0000","last_change":"2023-10-16T08:38:44.655693+0000","last_active":"2023-10-16T08:39:25.956207+0000","last_peered":"2023-10-16T08:39:25.956207+0000","last_clean":"2023-10-16T08:39:25.956207+0000","last_became_active":"2023-10-16T08:38:44.655001+0000","last_became_peered":"2023-10-16T08:38:44.655001+0000","last_unstale":"2023-10-16T08:39:25.956207+0000","last_undegraded":"2023-10-16T08:39:25.956207+0000","last_fullsized":"2023-10-16T08:39:25.956207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:30.649639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945004+0000","last_change":"2023-10-16T08:38:45.639658+0000","last_active":"2023-10-16T08:39:25.945004+0000","last_peered":"2023-10-16T08:39:25.945004+0000","last_clean":"2023-10-16T08:39:25.945004+0000","last_became_active":"2023-10-16T08:38:45.638702+0000","last_became_peered":"2023-10-16T08:38:45.638702+0000","last_unstale":"2023-10-16T08:39:25.945004+0000","last_undegraded":"2023-10-16T08:39:25.945004+0000","last_fullsized":"2023-10-16T08:39:25.945004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945364+0000","last_change":"2023-10-16T08:38:44.648715+0000","last_active":"2023-10-16T08:39:25.945364+0000","last_peered":"2023-10-16T08:39:25.945364+0000","last_clean":"2023-10-16T08:39:25.945364+0000","last_became_active":"2023-10-16T08:38:44.648312+0000","last_became_peered":"2023-10-16T08:38:44.648312+0000","last_unstale":"2023-10-16T08:39:25.945364+0000","last_undegraded":"2023-10-16T08:39:25.945364+0000","last_fullsized":"2023-10-16T08:39:25.945364+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955768+0000","last_change":"2023-10-16T08:38:45.663402+0000","last_active":"2023-10-16T08:39:25.955768+0000","last_peered":"2023-10-16T08:39:25.955768+0000","last_clean":"2023-10-16T08:39:25.955768+0000","last_became_active":"2023-10-16T08:38:45.662442+0000","last_became_peered":"2023-10-16T08:38:45.662442+0000","last_unstale":"2023-10-16T08:39:25.955768+0000","last_undegraded":"2023-10-16T08:39:25.955768+0000","last_fullsized":"2023-10-16T08:39:25.955768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949981+0000","last_change":"2023-10-16T08:38:44.650015+0000","last_active":"2023-10-16T08:39:25.949981+0000","last_peered":"2023-10-16T08:39:25.949981+0000","last_clean":"2023-10-16T08:39:25.949981+0000","last_became_active":"2023-10-16T08:38:44.648155+0000","last_became_peered":"2023-10-16T08:38:44.648155+0000","last_unstale":"2023-10-16T08:39:25.949981+0000","last_undegraded":"2023-10-16T08:39:25.949981+0000","last_fullsized":"2023-10-16T08:39:25.949981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402266+0000","last_change":"2023-10-16T08:38:45.637568+0000","last_active":"2023-10-16T08:39:26.402266+0000","last_peered":"2023-10-16T08:39:26.402266+0000","last_clean":"2023-10-16T08:39:26.402266+0000","last_became_active":"2023-10-16T08:38:45.636751+0000","last_became_peered":"2023-10-16T08:38:45.636751+0000","last_unstale":"2023-10-16T08:39:26.402266+0000","last_undegraded":"2023-10-16T08:39:26.402266+0000","last_fullsized":"2023-10-16T08:39:26.402266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954444+0000","last_change":"2023-10-16T08:38:44.649257+0000","last_active":"2023-10-16T08:39:25.954444+0000","last_peered":"2023-10-16T08:39:25.954444+0000","last_clean":"2023-10-16T08:39:25.954444+0000","last_became_active":"2023-10-16T08:38:44.646990+0000","last_became_peered":"2023-10-16T08:38:44.646990+0000","last_unstale":"2023-10-16T08:39:25.954444+0000","last_undegraded":"2023-10-16T08:39:25.954444+0000","last_fullsized":"2023-10-16T08:39:25.954444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403442+0000","last_change":"2023-10-16T08:38:45.648228+0000","last_active":"2023-10-16T08:39:26.403442+0000","last_peered":"2023-10-16T08:39:26.403442+0000","last_clean":"2023-10-16T08:39:26.403442+0000","last_became_active":"2023-10-16T08:38:45.647449+0000","last_became_peered":"2023-10-16T08:38:45.647449+0000","last_unstale":"2023-10-16T08:39:26.403442+0000","last_undegraded":"2023-10-16T08:39:26.403442+0000","last_fullsized":"2023-10-16T08:39:26.403442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:20.779617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.944878+0000","last_change":"2023-10-16T08:38:44.631354+0000","last_active":"2023-10-16T08:39:25.944878+0000","last_peered":"2023-10-16T08:39:25.944878+0000","last_clean":"2023-10-16T08:39:25.944878+0000","last_became_active":"2023-10-16T08:38:44.630451+0000","last_became_peered":"2023-10-16T08:38:44.630451+0000","last_unstale":"2023-10-16T08:39:25.944878+0000","last_undegraded":"2023-10-16T08:39:25.944878+0000","last_fullsized":"2023-10-16T08:39:25.944878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:10.980982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955719+0000","last_change":"2023-10-16T08:38:45.663898+0000","last_active":"2023-10-16T08:39:25.955719+0000","last_peered":"2023-10-16T08:39:25.955719+0000","last_clean":"2023-10-16T08:39:25.955719+0000","last_became_active":"2023-10-16T08:38:45.663100+0000","last_became_peered":"2023-10-16T08:38:45.663100+0000","last_unstale":"2023-10-16T08:39:25.955719+0000","last_undegraded":"2023-10-16T08:39:25.955719+0000","last_fullsized":"2023-10-16T08:39:25.955719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945128+0000","last_change":"2023-10-16T08:38:44.646987+0000","last_active":"2023-10-16T08:39:25.945128+0000","last_peered":"2023-10-16T08:39:25.945128+0000","last_clean":"2023-10-16T08:39:25.945128+0000","last_became_active":"2023-10-16T08:38:44.646580+0000","last_became_peered":"2023-10-16T08:38:44.646580+0000","last_unstale":"2023-10-16T08:39:25.945128+0000","last_undegraded":"2023-10-16T08:39:25.945128+0000","last_fullsized":"2023-10-16T08:39:25.945128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956260+0000","last_change":"2023-10-16T08:38:45.662894+0000","last_active":"2023-10-16T08:39:25.956260+0000","last_peered":"2023-10-16T08:39:25.956260+0000","last_clean":"2023-10-16T08:39:25.956260+0000","last_became_active":"2023-10-16T08:38:45.662226+0000","last_became_peered":"2023-10-16T08:38:45.662226+0000","last_unstale":"2023-10-16T08:39:25.956260+0000","last_undegraded":"2023-10-16T08:39:25.956260+0000","last_fullsized":"2023-10-16T08:39:25.956260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955552+0000","last_change":"2023-10-16T08:38:44.660571+0000","last_active":"2023-10-16T08:39:25.955552+0000","last_peered":"2023-10-16T08:39:25.955552+0000","last_clean":"2023-10-16T08:39:25.955552+0000","last_became_active":"2023-10-16T08:38:44.659976+0000","last_became_peered":"2023-10-16T08:38:44.659976+0000","last_unstale":"2023-10-16T08:39:25.955552+0000","last_undegraded":"2023-10-16T08:39:25.955552+0000","last_fullsized":"2023-10-16T08:39:25.955552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949392+0000","last_change":"2023-10-16T08:38:45.665200+0000","last_active":"2023-10-16T08:39:25.949392+0000","last_peered":"2023-10-16T08:39:25.949392+0000","last_clean":"2023-10-16T08:39:25.949392+0000","last_became_active":"2023-10-16T08:38:45.664616+0000","last_became_peered":"2023-10-16T08:38:45.664616+0000","last_unstale":"2023-10-16T08:39:25.949392+0000","last_undegraded":"2023-10-16T08:39:25.949392+0000","last_fullsized":"2023-10-16T08:39:25.949392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.953546+0000","last_change":"2023-10-16T08:38:45.680892+0000","last_active":"2023-10-16T08:39:25.953546+0000","last_peered":"2023-10-16T08:39:25.953546+0000","last_clean":"2023-10-16T08:39:25.953546+0000","last_became_active":"2023-10-16T08:38:45.680231+0000","last_became_peered":"2023-10-16T08:38:45.680231+0000","last_unstale":"2023-10-16T08:39:25.953546+0000","last_undegraded":"2023-10-16T08:39:25.953546+0000","last_fullsized":"2023-10-16T08:39:25.953546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:58:51.599709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956015+0000","last_change":"2023-10-16T08:38:44.664269+0000","last_active":"2023-10-16T08:39:25.956015+0000","last_peered":"2023-10-16T08:39:25.956015+0000","last_clean":"2023-10-16T08:39:25.956015+0000","last_became_active":"2023-10-16T08:38:44.663674+0000","last_became_peered":"2023-10-16T08:38:44.663674+0000","last_unstale":"2023-10-16T08:39:25.956015+0000","last_undegraded":"2023-10-16T08:39:25.956015+0000","last_fullsized":"2023-10-16T08:39:25.956015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951695+0000","last_change":"2023-10-16T08:38:45.680890+0000","last_active":"2023-10-16T08:39:25.951695+0000","last_peered":"2023-10-16T08:39:25.951695+0000","last_clean":"2023-10-16T08:39:25.951695+0000","last_became_active":"2023-10-16T08:38:45.680266+0000","last_became_peered":"2023-10-16T08:38:45.680266+0000","last_unstale":"2023-10-16T08:39:25.951695+0000","last_undegraded":"2023-10-16T08:39:25.951695+0000","last_fullsized":"2023-10-16T08:39:25.951695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:26.292137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955778+0000","last_change":"2023-10-16T08:38:44.653447+0000","last_active":"2023-10-16T08:39:25.955778+0000","last_peered":"2023-10-16T08:39:25.955778+0000","last_clean":"2023-10-16T08:39:25.955778+0000","last_became_active":"2023-10-16T08:38:44.652658+0000","last_became_peered":"2023-10-16T08:38:44.652658+0000","last_unstale":"2023-10-16T08:39:25.955778+0000","last_undegraded":"2023-10-16T08:39:25.955778+0000","last_fullsized":"2023-10-16T08:39:25.955778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:22.350066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402179+0000","last_change":"2023-10-16T08:38:45.665041+0000","last_active":"2023-10-16T08:39:26.402179+0000","last_peered":"2023-10-16T08:39:26.402179+0000","last_clean":"2023-10-16T08:39:26.402179+0000","last_became_active":"2023-10-16T08:38:45.664274+0000","last_became_peered":"2023-10-16T08:38:45.664274+0000","last_unstale":"2023-10-16T08:39:26.402179+0000","last_undegraded":"2023-10-16T08:39:26.402179+0000","last_fullsized":"2023-10-16T08:39:26.402179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955588+0000","last_change":"2023-10-16T08:38:44.645754+0000","last_active":"2023-10-16T08:39:25.955588+0000","last_peered":"2023-10-16T08:39:25.955588+0000","last_clean":"2023-10-16T08:39:25.955588+0000","last_became_active":"2023-10-16T08:38:44.645000+0000","last_became_peered":"2023-10-16T08:38:44.645000+0000","last_unstale":"2023-10-16T08:39:25.955588+0000","last_undegraded":"2023-10-16T08:39:25.955588+0000","last_fullsized":"2023-10-16T08:39:25.955588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955495+0000","last_change":"2023-10-16T08:38:45.664252+0000","last_active":"2023-10-16T08:39:25.955495+0000","last_peered":"2023-10-16T08:39:25.955495+0000","last_clean":"2023-10-16T08:39:25.955495+0000","last_became_active":"2023-10-16T08:38:45.663538+0000","last_became_peered":"2023-10-16T08:38:45.663538+0000","last_unstale":"2023-10-16T08:39:25.955495+0000","last_undegraded":"2023-10-16T08:39:25.955495+0000","last_fullsized":"2023-10-16T08:39:25.955495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402379+0000","last_change":"2023-10-16T08:38:44.646321+0000","last_active":"2023-10-16T08:39:26.402379+0000","last_peered":"2023-10-16T08:39:26.402379+0000","last_clean":"2023-10-16T08:39:26.402379+0000","last_became_active":"2023-10-16T08:38:44.645854+0000","last_became_peered":"2023-10-16T08:38:44.645854+0000","last_unstale":"2023-10-16T08:39:26.402379+0000","last_undegraded":"2023-10-16T08:39:26.402379+0000","last_fullsized":"2023-10-16T08:39:26.402379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:10.980982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6065,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":12130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":827392,"data_stored":674262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":1.3220000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.97699999999999998}]},{"osd":2,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":1.228}]},{"osd":3,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":1.363}]},{"osd":4,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":1.0569999999999999}]},{"osd":5,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.276},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":1.0900000000000001}]},{"osd":6,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":1.01}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":88579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.57499999999999996},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":0.63600000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":0.70599999999999996}]},{"osd":2,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.94099999999999995}]},{"osd":3,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":1.1930000000000001}]},{"osd":4,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":1.25}]},{"osd":5,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.3600000000000001}]},{"osd":7,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.04,"5min":2.04,"15min":2.04},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.3029999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":83402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.87}]},{"osd":1,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.165},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":1.0269999999999999}]},{"osd":2,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.133}]},{"osd":3,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.087},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.198}]},{"osd":4,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.98899999999999999}]},{"osd":6,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":0.84399999999999997}]},{"osd":7,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.111}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":83438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.004}]},{"osd":2,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.93799999999999994}]},{"osd":3,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.873}]},{"osd":5,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":1.05},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.754}]},{"osd":6,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":1.1060000000000001}]},{"osd":7,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.153,"5min":2.153,"15min":2.153},"last":1.079}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":88579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.51600000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.64400000000000002}]},{"osd":2,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.73099999999999998}]},{"osd":4,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.77400000000000002}]},{"osd":5,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.82699999999999996}]},{"osd":6,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":0.85999999999999999}]},{"osd":7,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.995}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":83312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.64700000000000002}]},{"osd":1,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.752}]},{"osd":3,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.79000000000000004}]},{"osd":4,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.119},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.93999999999999995}]},{"osd":5,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.23}]},{"osd":6,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.056},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":1.0840000000000001}]},{"osd":7,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.1970000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":83472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.48399999999999999}]},{"osd":2,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.68899999999999995}]},{"osd":3,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.60499999999999998}]},{"osd":4,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.70599999999999996}]},{"osd":5,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.123}]},{"osd":6,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.0049999999999999}]},{"osd":7,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.046},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.5449999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.94999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":835584,"data_stored":674318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.61499999999999999}]},{"osd":2,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.72199999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.56200000000000006}]},{"osd":4,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.75900000000000001}]},{"osd":5,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.97599999999999998}]},{"osd":6,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.91700000000000004}]},{"osd":7,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":1.0569999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:39:42.880 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-16T08:39:42.881 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:39:43.045 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.039+0000 7f40a6a58640 1 Processor -- start 2023-10-16T08:39:43.046 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.039+0000 7f40a6a58640 1 -- start start 2023-10-16T08:39:43.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.039+0000 7f40a6a58640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f40a015d180 0x7f40a006b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:43.047 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.039+0000 7f40a6a58640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f40a015d560 0x7f40a006fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:43.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.039+0000 7f40a6a58640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40a0070090 con 0x7f40a00ad820 2023-10-16T08:39:43.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.039+0000 7f40a6a58640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a00cf480 con 0x7f40a015d560 2023-10-16T08:39:43.048 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.039+0000 7f40a6a58640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40a00cf760 con 0x7f40a015d180 2023-10-16T08:39:43.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.039+0000 7f409f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f40a015d560 0x7f40a006fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:43.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.039+0000 7f409f7fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f40a015d560 0x7f40a006fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50732/0 (socket says 172.21.15.40:50732) 2023-10-16T08:39:43.049 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.039+0000 7f409f7fe640 1 -- 172.21.15.40:0/1233021990 learned_addr learned my addr 172.21.15.40:0/1233021990 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:43.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.039+0000 7f409ffff640 1 --2- 172.21.15.40:0/1233021990 >> v2:172.21.15.86:3300/0 conn(0x7f40a015d180 0x7f40a006b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:43.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.043+0000 7f409f7fe640 1 -- 172.21.15.40:0/1233021990 >> v2:172.21.15.86:3300/0 conn(0x7f40a015d180 msgr2=0x7f40a006b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:43.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.043+0000 7f409f7fe640 1 --2- 172.21.15.40:0/1233021990 >> v2:172.21.15.86:3300/0 conn(0x7f40a015d180 0x7f40a006b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:43.050 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.043+0000 7f409f7fe640 1 -- 172.21.15.40:0/1233021990 >> v1:172.21.15.86:6789/0 conn(0x7f40a00ad820 legacy=0x7f40a00c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:43.051 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.043+0000 7f409f7fe640 1 -- 172.21.15.40:0/1233021990 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40a00c8fb0 con 0x7f40a015d560 2023-10-16T08:39:43.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.043+0000 7f409f7fe640 1 --2- 172.21.15.40:0/1233021990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a015d560 0x7f40a006fb50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f40a006a0d0 tx=0x7f409001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=441b076e849efde6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:43.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.043+0000 7f409effd640 1 -- 172.21.15.40:0/1233021990 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40900ab670 con 0x7f40a015d560 2023-10-16T08:39:43.055 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.043+0000 7f409effd640 1 -- 172.21.15.40:0/1233021990 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40900ab7d0 con 0x7f40a015d560 2023-10-16T08:39:43.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.043+0000 7f409effd640 1 -- 172.21.15.40:0/1233021990 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40900abb60 con 0x7f40a015d560 2023-10-16T08:39:43.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.043+0000 7f40a6a58640 1 -- 172.21.15.40:0/1233021990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a015d560 msgr2=0x7f40a006fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:43.056 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.043+0000 7f40a6a58640 1 --2- 172.21.15.40:0/1233021990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a015d560 0x7f40a006fb50 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f40a006a0d0 tx=0x7f409001c490 comp rx=0 tx=0).stop 2023-10-16T08:39:43.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.043+0000 7f40a6a58640 1 -- 172.21.15.40:0/1233021990 shutdown_connections 2023-10-16T08:39:43.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.043+0000 7f40a6a58640 1 --2- 172.21.15.40:0/1233021990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a015d560 0x7f40a006fb50 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:43.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.043+0000 7f40a6a58640 1 --2- 172.21.15.40:0/1233021990 >> v2:172.21.15.86:3300/0 conn(0x7f40a015d180 0x7f40a006b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:43.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.043+0000 7f40a6a58640 1 -- 172.21.15.40:0/1233021990 >> 172.21.15.40:0/1233021990 conn(0x7f40a00ce960 msgr2=0x7f40a00ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:43.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.043+0000 7f40a6a58640 1 -- 172.21.15.40:0/1233021990 shutdown_connections 2023-10-16T08:39:43.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.043+0000 7f40a6a58640 1 -- 172.21.15.40:0/1233021990 wait complete. 2023-10-16T08:39:43.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f40a6a58640 1 Processor -- start 2023-10-16T08:39:43.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f40a6a58640 1 -- start start 2023-10-16T08:39:43.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f40a6a58640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40a00ad820 0x7f40a00ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:43.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f40a6a58640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40a015d180 0x7f40a00ace20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:43.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f40a6a58640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a015d560 0x7f40a006bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:43.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f40a6a58640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00caa40 con 0x7f40a015d560 2023-10-16T08:39:43.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f40a6a58640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00b1720 con 0x7f40a015d180 2023-10-16T08:39:43.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f40a6a58640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40a00b1a00 con 0x7f40a00ad820 2023-10-16T08:39:43.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f40a4ff5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a015d560 0x7f40a006bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:43.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f40a4ff5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a015d560 0x7f40a006bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50744/0 (socket says 172.21.15.40:50744) 2023-10-16T08:39:43.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f40a4ff5640 1 -- 172.21.15.40:0/3111782252 learned_addr learned my addr 172.21.15.40:0/3111782252 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:43.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f409f7fe640 1 --2- 172.21.15.40:0/3111782252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40a015d180 0x7f40a00ace20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:43.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f409ffff640 1 --2- 172.21.15.40:0/3111782252 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40a00ad820 0x7f40a00ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:43.063 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f40a4ff5640 1 -- 172.21.15.40:0/3111782252 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40a00ad820 msgr2=0x7f40a00ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:43.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f40a4ff5640 1 --2- 172.21.15.40:0/3111782252 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40a00ad820 0x7f40a00ac8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:43.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f40a4ff5640 1 -- 172.21.15.40:0/3111782252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40a015d180 msgr2=0x7f40a00ace20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:43.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f40a4ff5640 1 --2- 172.21.15.40:0/3111782252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40a015d180 0x7f40a00ace20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:43.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f40a4ff5640 1 -- 172.21.15.40:0/3111782252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40900094d0 con 0x7f40a015d560 2023-10-16T08:39:43.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f409f7fe640 1 --2- 172.21.15.40:0/3111782252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40a015d180 0x7f40a00ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:43.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f40a4ff5640 1 --2- 172.21.15.40:0/3111782252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a015d560 0x7f40a006bdd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f409800c780 tx=0x7f409800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:43.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f409cff9640 1 -- 172.21.15.40:0/3111782252 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4098004b20 con 0x7f40a015d560 2023-10-16T08:39:43.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f409cff9640 1 -- 172.21.15.40:0/3111782252 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4098004c80 con 0x7f40a015d560 2023-10-16T08:39:43.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f409cff9640 1 -- 172.21.15.40:0/3111782252 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f409800aa90 con 0x7f40a015d560 2023-10-16T08:39:43.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f40a6a58640 1 -- 172.21.15.40:0/3111782252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40a00b1ce0 con 0x7f40a015d560 2023-10-16T08:39:43.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f40a6a58640 1 -- 172.21.15.40:0/3111782252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40a00ca680 con 0x7f40a015d560 2023-10-16T08:39:43.068 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.047+0000 7f40a6a58640 1 -- 172.21.15.40:0/3111782252 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40a004e9c0 con 0x7f40a015d560 2023-10-16T08:39:43.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.051+0000 7f409cff9640 1 -- 172.21.15.40:0/3111782252 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f409800abf0 con 0x7f40a015d560 2023-10-16T08:39:43.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.051+0000 7f409cff9640 1 --2- 172.21.15.40:0/3111782252 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f40800664a0 0x7f4080068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:43.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.051+0000 7f409cff9640 1 -- 172.21.15.40:0/3111782252 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f4098083f60 con 0x7f40a015d560 2023-10-16T08:39:43.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.051+0000 7f409ffff640 1 --2- 172.21.15.40:0/3111782252 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f40800664a0 0x7f4080068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:43.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.051+0000 7f409ffff640 1 --2- 172.21.15.40:0/3111782252 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f40800664a0 0x7f4080068960 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f408800b9d0 tx=0x7f4088008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:43.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.055+0000 7f409cff9640 1 -- 172.21.15.40:0/3111782252 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f40980513e0 con 0x7f40a015d560 2023-10-16T08:39:43.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.211+0000 7f40a6a58640 1 -- 172.21.15.40:0/3111782252 --> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f40a00b1890 con 0x7f40800664a0 2023-10-16T08:39:43.238 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.235+0000 7f409cff9640 1 -- 172.21.15.40:0/3111782252 <== mgr.4111 v2:172.21.15.86:6832/1188970899 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286974 (secure 0 0 0) 0x7f40a00b1890 con 0x7f40800664a0 2023-10-16T08:39:43.241 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:39:43.243 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-16T08:39:43.244 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.235+0000 7f40a6a58640 1 -- 172.21.15.40:0/3111782252 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f40800664a0 msgr2=0x7f4080068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:43.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.235+0000 7f40a6a58640 1 --2- 172.21.15.40:0/3111782252 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f40800664a0 0x7f4080068960 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f408800b9d0 tx=0x7f4088008040 comp rx=0 tx=0).stop 2023-10-16T08:39:43.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.235+0000 7f40a6a58640 1 -- 172.21.15.40:0/3111782252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a015d560 msgr2=0x7f40a006bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:43.245 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.235+0000 7f40a6a58640 1 --2- 172.21.15.40:0/3111782252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a015d560 0x7f40a006bdd0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f409800c780 tx=0x7f409800cc40 comp rx=0 tx=0).stop 2023-10-16T08:39:43.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.235+0000 7f40a6a58640 1 -- 172.21.15.40:0/3111782252 shutdown_connections 2023-10-16T08:39:43.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.235+0000 7f40a6a58640 1 --2- 172.21.15.40:0/3111782252 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f40800664a0 0x7f4080068960 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:43.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.235+0000 7f40a6a58640 1 --2- 172.21.15.40:0/3111782252 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40a015d560 0x7f40a006bdd0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:43.246 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.235+0000 7f40a6a58640 1 --2- 172.21.15.40:0/3111782252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40a015d180 0x7f40a00ace20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:43.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.235+0000 7f40a6a58640 1 --2- 172.21.15.40:0/3111782252 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40a00ad820 0x7f40a00ac8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:43.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.235+0000 7f40a6a58640 1 -- 172.21.15.40:0/3111782252 >> 172.21.15.40:0/3111782252 conn(0x7f40a00ce960 msgr2=0x7f40a0070ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:43.247 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.235+0000 7f40a6a58640 1 -- 172.21.15.40:0/3111782252 shutdown_connections 2023-10-16T08:39:43.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.235+0000 7f40a6a58640 1 -- 172.21.15.40:0/3111782252 wait complete. 2023-10-16T08:39:43.261 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":50,"stamp":"2023-10-16T08:39:42.958465+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596433,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":218,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":169,"ondisk_log_size":169,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219200,"kb_used_data":3024,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755520,"statfs":{"total":773094113280,"available":772869652480,"internally_reserved":0,"allocated":3096576,"data_stored":1859362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004172"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.964601+0000","last_change":"2023-10-16T08:39:26.963087+0000","last_active":"2023-10-16T08:39:26.964601+0000","last_peered":"2023-10-16T08:39:26.964601+0000","last_clean":"2023-10-16T08:39:26.964601+0000","last_became_active":"2023-10-16T08:39:26.962446+0000","last_became_peered":"2023-10-16T08:39:26.962446+0000","last_unstale":"2023-10-16T08:39:26.964601+0000","last_undegraded":"2023-10-16T08:39:26.964601+0000","last_fullsized":"2023-10-16T08:39:26.964601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:33.190218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945446+0000","last_change":"2023-10-16T08:38:45.657659+0000","last_active":"2023-10-16T08:39:25.945446+0000","last_peered":"2023-10-16T08:39:25.945446+0000","last_clean":"2023-10-16T08:39:25.945446+0000","last_became_active":"2023-10-16T08:38:45.656884+0000","last_became_peered":"2023-10-16T08:38:45.656884+0000","last_unstale":"2023-10-16T08:39:25.945446+0000","last_undegraded":"2023-10-16T08:39:25.945446+0000","last_fullsized":"2023-10-16T08:39:25.945446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402116+0000","last_change":"2023-10-16T08:38:45.630125+0000","last_active":"2023-10-16T08:39:26.402116+0000","last_peered":"2023-10-16T08:39:26.402116+0000","last_clean":"2023-10-16T08:39:26.402116+0000","last_became_active":"2023-10-16T08:38:45.629297+0000","last_became_peered":"2023-10-16T08:38:45.629297+0000","last_unstale":"2023-10-16T08:39:26.402116+0000","last_undegraded":"2023-10-16T08:39:26.402116+0000","last_fullsized":"2023-10-16T08:39:26.402116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951941+0000","last_change":"2023-10-16T08:38:45.675996+0000","last_active":"2023-10-16T08:39:25.951941+0000","last_peered":"2023-10-16T08:39:25.951941+0000","last_clean":"2023-10-16T08:39:25.951941+0000","last_became_active":"2023-10-16T08:38:45.675327+0000","last_became_peered":"2023-10-16T08:38:45.675327+0000","last_unstale":"2023-10-16T08:39:25.951941+0000","last_undegraded":"2023-10-16T08:39:25.951941+0000","last_fullsized":"2023-10-16T08:39:25.951941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:33.190218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955358+0000","last_change":"2023-10-16T08:38:45.653550+0000","last_active":"2023-10-16T08:39:25.955358+0000","last_peered":"2023-10-16T08:39:25.955358+0000","last_clean":"2023-10-16T08:39:25.955358+0000","last_became_active":"2023-10-16T08:38:45.652808+0000","last_became_peered":"2023-10-16T08:38:45.652808+0000","last_unstale":"2023-10-16T08:39:25.955358+0000","last_undegraded":"2023-10-16T08:39:25.955358+0000","last_fullsized":"2023-10-16T08:39:25.955358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.950945+0000","last_change":"2023-10-16T08:38:45.675086+0000","last_active":"2023-10-16T08:39:25.950945+0000","last_peered":"2023-10-16T08:39:25.950945+0000","last_clean":"2023-10-16T08:39:25.950945+0000","last_became_active":"2023-10-16T08:38:45.671146+0000","last_became_peered":"2023-10-16T08:38:45.671146+0000","last_unstale":"2023-10-16T08:39:25.950945+0000","last_undegraded":"2023-10-16T08:39:25.950945+0000","last_fullsized":"2023-10-16T08:39:25.950945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954769+0000","last_change":"2023-10-16T08:38:45.652595+0000","last_active":"2023-10-16T08:39:25.954769+0000","last_peered":"2023-10-16T08:39:25.954769+0000","last_clean":"2023-10-16T08:39:25.954769+0000","last_became_active":"2023-10-16T08:38:45.651848+0000","last_became_peered":"2023-10-16T08:38:45.651848+0000","last_unstale":"2023-10-16T08:39:25.954769+0000","last_undegraded":"2023-10-16T08:39:25.954769+0000","last_fullsized":"2023-10-16T08:39:25.954769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:13.453321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955228+0000","last_change":"2023-10-16T08:38:45.668858+0000","last_active":"2023-10-16T08:39:25.955228+0000","last_peered":"2023-10-16T08:39:25.955228+0000","last_clean":"2023-10-16T08:39:25.955228+0000","last_became_active":"2023-10-16T08:38:45.664702+0000","last_became_peered":"2023-10-16T08:38:45.664702+0000","last_unstale":"2023-10-16T08:39:25.955228+0000","last_undegraded":"2023-10-16T08:39:25.955228+0000","last_fullsized":"2023-10-16T08:39:25.955228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955879+0000","last_change":"2023-10-16T08:38:45.667037+0000","last_active":"2023-10-16T08:39:25.955879+0000","last_peered":"2023-10-16T08:39:25.955879+0000","last_clean":"2023-10-16T08:39:25.955879+0000","last_became_active":"2023-10-16T08:38:45.666474+0000","last_became_peered":"2023-10-16T08:38:45.666474+0000","last_unstale":"2023-10-16T08:39:25.955879+0000","last_undegraded":"2023-10-16T08:39:25.955879+0000","last_fullsized":"2023-10-16T08:39:25.955879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:11:02.634006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949980+0000","last_change":"2023-10-16T08:38:45.669859+0000","last_active":"2023-10-16T08:39:25.949980+0000","last_peered":"2023-10-16T08:39:25.949980+0000","last_clean":"2023-10-16T08:39:25.949980+0000","last_became_active":"2023-10-16T08:38:45.668278+0000","last_became_peered":"2023-10-16T08:38:45.668278+0000","last_unstale":"2023-10-16T08:39:25.949980+0000","last_undegraded":"2023-10-16T08:39:25.949980+0000","last_fullsized":"2023-10-16T08:39:25.949980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949617+0000","last_change":"2023-10-16T08:38:45.664567+0000","last_active":"2023-10-16T08:39:25.949617+0000","last_peered":"2023-10-16T08:39:25.949617+0000","last_clean":"2023-10-16T08:39:25.949617+0000","last_became_active":"2023-10-16T08:38:45.663813+0000","last_became_peered":"2023-10-16T08:38:45.663813+0000","last_unstale":"2023-10-16T08:39:25.949617+0000","last_undegraded":"2023-10-16T08:39:25.949617+0000","last_fullsized":"2023-10-16T08:39:25.949617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403718+0000","last_change":"2023-10-16T08:38:45.675925+0000","last_active":"2023-10-16T08:39:26.403718+0000","last_peered":"2023-10-16T08:39:26.403718+0000","last_clean":"2023-10-16T08:39:26.403718+0000","last_became_active":"2023-10-16T08:38:45.675320+0000","last_became_peered":"2023-10-16T08:38:45.675320+0000","last_unstale":"2023-10-16T08:39:26.403718+0000","last_undegraded":"2023-10-16T08:39:26.403718+0000","last_fullsized":"2023-10-16T08:39:26.403718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:11.991837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956028+0000","last_change":"2023-10-16T08:38:45.668056+0000","last_active":"2023-10-16T08:39:25.956028+0000","last_peered":"2023-10-16T08:39:25.956028+0000","last_clean":"2023-10-16T08:39:25.956028+0000","last_became_active":"2023-10-16T08:38:45.667428+0000","last_became_peered":"2023-10-16T08:38:45.667428+0000","last_unstale":"2023-10-16T08:39:25.956028+0000","last_undegraded":"2023-10-16T08:39:25.956028+0000","last_fullsized":"2023-10-16T08:39:25.956028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.401718+0000","last_change":"2023-10-16T08:38:45.637518+0000","last_active":"2023-10-16T08:39:26.401718+0000","last_peered":"2023-10-16T08:39:26.401718+0000","last_clean":"2023-10-16T08:39:26.401718+0000","last_became_active":"2023-10-16T08:38:45.636785+0000","last_became_peered":"2023-10-16T08:38:45.636785+0000","last_unstale":"2023-10-16T08:39:26.401718+0000","last_undegraded":"2023-10-16T08:39:26.401718+0000","last_fullsized":"2023-10-16T08:39:26.401718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:11.870171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.944872+0000","last_change":"2023-10-16T08:38:45.656433+0000","last_active":"2023-10-16T08:39:25.944872+0000","last_peered":"2023-10-16T08:39:25.944872+0000","last_clean":"2023-10-16T08:39:25.944872+0000","last_became_active":"2023-10-16T08:38:45.655395+0000","last_became_peered":"2023-10-16T08:38:45.655395+0000","last_unstale":"2023-10-16T08:39:25.944872+0000","last_undegraded":"2023-10-16T08:39:25.944872+0000","last_fullsized":"2023-10-16T08:39:25.944872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.950540+0000","last_change":"2023-10-16T08:38:45.664192+0000","last_active":"2023-10-16T08:39:25.950540+0000","last_peered":"2023-10-16T08:39:25.950540+0000","last_clean":"2023-10-16T08:39:25.950540+0000","last_became_active":"2023-10-16T08:38:45.663312+0000","last_became_peered":"2023-10-16T08:38:45.663312+0000","last_unstale":"2023-10-16T08:39:25.950540+0000","last_undegraded":"2023-10-16T08:39:25.950540+0000","last_fullsized":"2023-10-16T08:39:25.950540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.016191+0000","last_change":"2023-10-16T08:39:27.013093+0000","last_active":"2023-10-16T08:39:27.016191+0000","last_peered":"2023-10-16T08:39:27.016191+0000","last_clean":"2023-10-16T08:39:27.016191+0000","last_became_active":"2023-10-16T08:39:27.012068+0000","last_became_peered":"2023-10-16T08:39:27.012068+0000","last_unstale":"2023-10-16T08:39:27.016191+0000","last_undegraded":"2023-10-16T08:39:27.016191+0000","last_fullsized":"2023-10-16T08:39:27.016191+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945605+0000","last_change":"2023-10-16T08:38:45.631488+0000","last_active":"2023-10-16T08:39:25.945605+0000","last_peered":"2023-10-16T08:39:25.945605+0000","last_clean":"2023-10-16T08:39:25.945605+0000","last_became_active":"2023-10-16T08:38:45.630641+0000","last_became_peered":"2023-10-16T08:38:45.630641+0000","last_unstale":"2023-10-16T08:39:25.945605+0000","last_undegraded":"2023-10-16T08:39:25.945605+0000","last_fullsized":"2023-10-16T08:39:25.945605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:11.991837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955988+0000","last_change":"2023-10-16T08:38:45.650087+0000","last_active":"2023-10-16T08:39:25.955988+0000","last_peered":"2023-10-16T08:39:25.955988+0000","last_clean":"2023-10-16T08:39:25.955988+0000","last_became_active":"2023-10-16T08:38:45.649475+0000","last_became_peered":"2023-10-16T08:38:45.649475+0000","last_unstale":"2023-10-16T08:39:25.955988+0000","last_undegraded":"2023-10-16T08:39:25.955988+0000","last_fullsized":"2023-10-16T08:39:25.955988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:26.292137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949532+0000","last_change":"2023-10-16T08:38:45.667336+0000","last_active":"2023-10-16T08:39:25.949532+0000","last_peered":"2023-10-16T08:39:25.949532+0000","last_clean":"2023-10-16T08:39:25.949532+0000","last_became_active":"2023-10-16T08:38:45.665291+0000","last_became_peered":"2023-10-16T08:38:45.665291+0000","last_unstale":"2023-10-16T08:39:25.949532+0000","last_undegraded":"2023-10-16T08:39:25.949532+0000","last_fullsized":"2023-10-16T08:39:25.949532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402191+0000","last_change":"2023-10-16T08:38:45.654036+0000","last_active":"2023-10-16T08:39:26.402191+0000","last_peered":"2023-10-16T08:39:26.402191+0000","last_clean":"2023-10-16T08:39:26.402191+0000","last_became_active":"2023-10-16T08:38:45.652412+0000","last_became_peered":"2023-10-16T08:38:45.652412+0000","last_unstale":"2023-10-16T08:39:26.402191+0000","last_undegraded":"2023-10-16T08:39:26.402191+0000","last_fullsized":"2023-10-16T08:39:26.402191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951156+0000","last_change":"2023-10-16T08:38:45.677644+0000","last_active":"2023-10-16T08:39:25.951156+0000","last_peered":"2023-10-16T08:39:25.951156+0000","last_clean":"2023-10-16T08:39:25.951156+0000","last_became_active":"2023-10-16T08:38:45.676844+0000","last_became_peered":"2023-10-16T08:38:45.676844+0000","last_unstale":"2023-10-16T08:39:25.951156+0000","last_undegraded":"2023-10-16T08:39:25.951156+0000","last_fullsized":"2023-10-16T08:39:25.951156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955256+0000","last_change":"2023-10-16T08:38:45.663771+0000","last_active":"2023-10-16T08:39:25.955256+0000","last_peered":"2023-10-16T08:39:25.955256+0000","last_clean":"2023-10-16T08:39:25.955256+0000","last_became_active":"2023-10-16T08:38:45.662944+0000","last_became_peered":"2023-10-16T08:38:45.662944+0000","last_unstale":"2023-10-16T08:39:25.955256+0000","last_undegraded":"2023-10-16T08:39:25.955256+0000","last_fullsized":"2023-10-16T08:39:25.955256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.950669+0000","last_change":"2023-10-16T08:38:45.667398+0000","last_active":"2023-10-16T08:39:25.950669+0000","last_peered":"2023-10-16T08:39:25.950669+0000","last_clean":"2023-10-16T08:39:25.950669+0000","last_became_active":"2023-10-16T08:38:45.666641+0000","last_became_peered":"2023-10-16T08:38:45.666641+0000","last_unstale":"2023-10-16T08:39:25.950669+0000","last_undegraded":"2023-10-16T08:39:25.950669+0000","last_fullsized":"2023-10-16T08:39:25.950669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955685+0000","last_change":"2023-10-16T08:38:45.670526+0000","last_active":"2023-10-16T08:39:25.955685+0000","last_peered":"2023-10-16T08:39:25.955685+0000","last_clean":"2023-10-16T08:39:25.955685+0000","last_became_active":"2023-10-16T08:38:45.669734+0000","last_became_peered":"2023-10-16T08:38:45.669734+0000","last_unstale":"2023-10-16T08:39:25.955685+0000","last_undegraded":"2023-10-16T08:39:25.955685+0000","last_fullsized":"2023-10-16T08:39:25.955685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949999+0000","last_change":"2023-10-16T08:38:45.670429+0000","last_active":"2023-10-16T08:39:25.949999+0000","last_peered":"2023-10-16T08:39:25.949999+0000","last_clean":"2023-10-16T08:39:25.949999+0000","last_became_active":"2023-10-16T08:38:45.665972+0000","last_became_peered":"2023-10-16T08:38:45.665972+0000","last_unstale":"2023-10-16T08:39:25.949999+0000","last_undegraded":"2023-10-16T08:39:25.949999+0000","last_fullsized":"2023-10-16T08:39:25.949999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945074+0000","last_change":"2023-10-16T08:38:45.667047+0000","last_active":"2023-10-16T08:39:25.945074+0000","last_peered":"2023-10-16T08:39:25.945074+0000","last_clean":"2023-10-16T08:39:25.945074+0000","last_became_active":"2023-10-16T08:38:45.666403+0000","last_became_peered":"2023-10-16T08:38:45.666403+0000","last_unstale":"2023-10-16T08:39:25.945074+0000","last_undegraded":"2023-10-16T08:39:25.945074+0000","last_fullsized":"2023-10-16T08:39:25.945074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956047+0000","last_change":"2023-10-16T08:38:45.668007+0000","last_active":"2023-10-16T08:39:25.956047+0000","last_peered":"2023-10-16T08:39:25.956047+0000","last_clean":"2023-10-16T08:39:25.956047+0000","last_became_active":"2023-10-16T08:38:45.664572+0000","last_became_peered":"2023-10-16T08:38:45.664572+0000","last_unstale":"2023-10-16T08:39:25.956047+0000","last_undegraded":"2023-10-16T08:39:25.956047+0000","last_fullsized":"2023-10-16T08:39:25.956047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402369+0000","last_change":"2023-10-16T08:38:45.667378+0000","last_active":"2023-10-16T08:39:26.402369+0000","last_peered":"2023-10-16T08:39:26.402369+0000","last_clean":"2023-10-16T08:39:26.402369+0000","last_became_active":"2023-10-16T08:38:45.666547+0000","last_became_peered":"2023-10-16T08:38:45.666547+0000","last_unstale":"2023-10-16T08:39:26.402369+0000","last_undegraded":"2023-10-16T08:39:26.402369+0000","last_fullsized":"2023-10-16T08:39:26.402369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.013174+0000","last_change":"2023-10-16T08:39:27.010902+0000","last_active":"2023-10-16T08:39:27.013174+0000","last_peered":"2023-10-16T08:39:27.013174+0000","last_clean":"2023-10-16T08:39:27.013174+0000","last_became_active":"2023-10-16T08:39:27.009983+0000","last_became_peered":"2023-10-16T08:39:27.009983+0000","last_unstale":"2023-10-16T08:39:27.013174+0000","last_undegraded":"2023-10-16T08:39:27.013174+0000","last_fullsized":"2023-10-16T08:39:27.013174+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.014469+0000","last_change":"2023-10-16T08:39:27.014469+0000","last_active":"2023-10-16T08:39:27.014469+0000","last_peered":"2023-10-16T08:39:27.014469+0000","last_clean":"2023-10-16T08:39:27.014469+0000","last_became_active":"2023-10-16T08:39:27.013658+0000","last_became_peered":"2023-10-16T08:39:27.013658+0000","last_unstale":"2023-10-16T08:39:27.014469+0000","last_undegraded":"2023-10-16T08:39:27.014469+0000","last_fullsized":"2023-10-16T08:39:27.014469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949746+0000","last_change":"2023-10-16T08:38:45.638183+0000","last_active":"2023-10-16T08:39:25.949746+0000","last_peered":"2023-10-16T08:39:25.949746+0000","last_clean":"2023-10-16T08:39:25.949746+0000","last_became_active":"2023-10-16T08:38:45.636762+0000","last_became_peered":"2023-10-16T08:38:45.636762+0000","last_unstale":"2023-10-16T08:39:25.949746+0000","last_undegraded":"2023-10-16T08:39:25.949746+0000","last_fullsized":"2023-10-16T08:39:25.949746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403351+0000","last_change":"2023-10-16T08:38:45.659464+0000","last_active":"2023-10-16T08:39:26.403351+0000","last_peered":"2023-10-16T08:39:26.403351+0000","last_clean":"2023-10-16T08:39:26.403351+0000","last_became_active":"2023-10-16T08:38:45.658808+0000","last_became_peered":"2023-10-16T08:38:45.658808+0000","last_unstale":"2023-10-16T08:39:26.403351+0000","last_undegraded":"2023-10-16T08:39:26.403351+0000","last_fullsized":"2023-10-16T08:39:26.403351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:25.386221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956227+0000","last_change":"2023-10-16T08:38:44.644122+0000","last_active":"2023-10-16T08:39:25.956227+0000","last_peered":"2023-10-16T08:39:25.956227+0000","last_clean":"2023-10-16T08:39:25.956227+0000","last_became_active":"2023-10-16T08:38:44.643614+0000","last_became_peered":"2023-10-16T08:38:44.643614+0000","last_unstale":"2023-10-16T08:39:25.956227+0000","last_undegraded":"2023-10-16T08:39:25.956227+0000","last_fullsized":"2023-10-16T08:39:25.956227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402402+0000","last_change":"2023-10-16T08:38:45.659854+0000","last_active":"2023-10-16T08:39:26.402402+0000","last_peered":"2023-10-16T08:39:26.402402+0000","last_clean":"2023-10-16T08:39:26.402402+0000","last_became_active":"2023-10-16T08:38:45.658409+0000","last_became_peered":"2023-10-16T08:38:45.658409+0000","last_unstale":"2023-10-16T08:39:26.402402+0000","last_undegraded":"2023-10-16T08:39:26.402402+0000","last_fullsized":"2023-10-16T08:39:26.402402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949532+0000","last_change":"2023-10-16T08:38:44.658819+0000","last_active":"2023-10-16T08:39:25.949532+0000","last_peered":"2023-10-16T08:39:25.949532+0000","last_clean":"2023-10-16T08:39:25.949532+0000","last_became_active":"2023-10-16T08:38:44.658257+0000","last_became_peered":"2023-10-16T08:38:44.658257+0000","last_unstale":"2023-10-16T08:39:25.949532+0000","last_undegraded":"2023-10-16T08:39:25.949532+0000","last_fullsized":"2023-10-16T08:39:25.949532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:54.848445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955337+0000","last_change":"2023-10-16T08:38:44.647536+0000","last_active":"2023-10-16T08:39:25.955337+0000","last_peered":"2023-10-16T08:39:25.955337+0000","last_clean":"2023-10-16T08:39:25.955337+0000","last_became_active":"2023-10-16T08:38:44.645931+0000","last_became_peered":"2023-10-16T08:38:44.645931+0000","last_unstale":"2023-10-16T08:39:25.955337+0000","last_undegraded":"2023-10-16T08:39:25.955337+0000","last_fullsized":"2023-10-16T08:39:25.955337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955491+0000","last_change":"2023-10-16T08:38:45.662855+0000","last_active":"2023-10-16T08:39:25.955491+0000","last_peered":"2023-10-16T08:39:25.955491+0000","last_clean":"2023-10-16T08:39:25.955491+0000","last_became_active":"2023-10-16T08:38:45.662072+0000","last_became_peered":"2023-10-16T08:38:45.662072+0000","last_unstale":"2023-10-16T08:39:25.955491+0000","last_undegraded":"2023-10-16T08:39:25.955491+0000","last_fullsized":"2023-10-16T08:39:25.955491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.950636+0000","last_change":"2023-10-16T08:38:44.657651+0000","last_active":"2023-10-16T08:39:25.950636+0000","last_peered":"2023-10-16T08:39:25.950636+0000","last_clean":"2023-10-16T08:39:25.950636+0000","last_became_active":"2023-10-16T08:38:44.657065+0000","last_became_peered":"2023-10-16T08:38:44.657065+0000","last_unstale":"2023-10-16T08:39:25.950636+0000","last_undegraded":"2023-10-16T08:39:25.950636+0000","last_fullsized":"2023-10-16T08:39:25.950636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.944769+0000","last_change":"2023-10-16T08:38:45.668048+0000","last_active":"2023-10-16T08:39:25.944769+0000","last_peered":"2023-10-16T08:39:25.944769+0000","last_clean":"2023-10-16T08:39:25.944769+0000","last_became_active":"2023-10-16T08:38:45.667413+0000","last_became_peered":"2023-10-16T08:38:45.667413+0000","last_unstale":"2023-10-16T08:39:25.944769+0000","last_undegraded":"2023-10-16T08:39:25.944769+0000","last_fullsized":"2023-10-16T08:39:25.944769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949842+0000","last_change":"2023-10-16T08:38:44.663188+0000","last_active":"2023-10-16T08:39:25.949842+0000","last_peered":"2023-10-16T08:39:25.949842+0000","last_clean":"2023-10-16T08:39:25.949842+0000","last_became_active":"2023-10-16T08:38:44.662383+0000","last_became_peered":"2023-10-16T08:38:44.662383+0000","last_unstale":"2023-10-16T08:39:25.949842+0000","last_undegraded":"2023-10-16T08:39:25.949842+0000","last_fullsized":"2023-10-16T08:39:25.949842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:24.375366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.953576+0000","last_change":"2023-10-16T08:38:45.679271+0000","last_active":"2023-10-16T08:39:25.953576+0000","last_peered":"2023-10-16T08:39:25.953576+0000","last_clean":"2023-10-16T08:39:25.953576+0000","last_became_active":"2023-10-16T08:38:45.678050+0000","last_became_peered":"2023-10-16T08:38:45.678050+0000","last_unstale":"2023-10-16T08:39:25.953576+0000","last_undegraded":"2023-10-16T08:39:25.953576+0000","last_fullsized":"2023-10-16T08:39:25.953576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.953752+0000","last_change":"2023-10-16T08:38:44.653478+0000","last_active":"2023-10-16T08:39:25.953752+0000","last_peered":"2023-10-16T08:39:25.953752+0000","last_clean":"2023-10-16T08:39:25.953752+0000","last_became_active":"2023-10-16T08:38:44.652618+0000","last_became_peered":"2023-10-16T08:38:44.652618+0000","last_unstale":"2023-10-16T08:39:25.953752+0000","last_undegraded":"2023-10-16T08:39:25.953752+0000","last_fullsized":"2023-10-16T08:39:25.953752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402370+0000","last_change":"2023-10-16T08:38:45.656603+0000","last_active":"2023-10-16T08:39:26.402370+0000","last_peered":"2023-10-16T08:39:26.402370+0000","last_clean":"2023-10-16T08:39:26.402370+0000","last_became_active":"2023-10-16T08:38:45.655213+0000","last_became_peered":"2023-10-16T08:38:45.655213+0000","last_unstale":"2023-10-16T08:39:26.402370+0000","last_undegraded":"2023-10-16T08:39:26.402370+0000","last_fullsized":"2023-10-16T08:39:26.402370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945084+0000","last_change":"2023-10-16T08:38:44.651637+0000","last_active":"2023-10-16T08:39:25.945084+0000","last_peered":"2023-10-16T08:39:25.945084+0000","last_clean":"2023-10-16T08:39:25.945084+0000","last_became_active":"2023-10-16T08:38:44.651359+0000","last_became_peered":"2023-10-16T08:38:44.651359+0000","last_unstale":"2023-10-16T08:39:25.945084+0000","last_undegraded":"2023-10-16T08:39:25.945084+0000","last_fullsized":"2023-10-16T08:39:25.945084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:10.859316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403856+0000","last_change":"2023-10-16T08:38:45.661741+0000","last_active":"2023-10-16T08:39:26.403856+0000","last_peered":"2023-10-16T08:39:26.403856+0000","last_clean":"2023-10-16T08:39:26.403856+0000","last_became_active":"2023-10-16T08:38:45.660611+0000","last_became_peered":"2023-10-16T08:38:45.660611+0000","last_unstale":"2023-10-16T08:39:26.403856+0000","last_undegraded":"2023-10-16T08:39:26.403856+0000","last_fullsized":"2023-10-16T08:39:26.403856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:55.859300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'18","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:35.375472+0000","last_change":"2023-10-16T08:38:44.647593+0000","last_active":"2023-10-16T08:39:35.375472+0000","last_peered":"2023-10-16T08:39:35.375472+0000","last_clean":"2023-10-16T08:39:35.375472+0000","last_became_active":"2023-10-16T08:38:44.647001+0000","last_became_peered":"2023-10-16T08:38:44.647001+0000","last_unstale":"2023-10-16T08:39:35.375472+0000","last_undegraded":"2023-10-16T08:39:35.375472+0000","last_fullsized":"2023-10-16T08:39:35.375472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5267,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955477+0000","last_change":"2023-10-16T08:38:45.668629+0000","last_active":"2023-10-16T08:39:25.955477+0000","last_peered":"2023-10-16T08:39:25.955477+0000","last_clean":"2023-10-16T08:39:25.955477+0000","last_became_active":"2023-10-16T08:38:45.667334+0000","last_became_peered":"2023-10-16T08:38:45.667334+0000","last_unstale":"2023-10-16T08:39:25.955477+0000","last_undegraded":"2023-10-16T08:39:25.955477+0000","last_fullsized":"2023-10-16T08:39:25.955477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402610+0000","last_change":"2023-10-16T08:38:44.655832+0000","last_active":"2023-10-16T08:39:26.402610+0000","last_peered":"2023-10-16T08:39:26.402610+0000","last_clean":"2023-10-16T08:39:26.402610+0000","last_became_active":"2023-10-16T08:38:44.655469+0000","last_became_peered":"2023-10-16T08:38:44.655469+0000","last_unstale":"2023-10-16T08:39:26.402610+0000","last_undegraded":"2023-10-16T08:39:26.402610+0000","last_fullsized":"2023-10-16T08:39:26.402610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:19.768762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.401855+0000","last_change":"2023-10-16T08:38:45.653688+0000","last_active":"2023-10-16T08:39:26.401855+0000","last_peered":"2023-10-16T08:39:26.401855+0000","last_clean":"2023-10-16T08:39:26.401855+0000","last_became_active":"2023-10-16T08:38:45.650563+0000","last_became_peered":"2023-10-16T08:38:45.650563+0000","last_unstale":"2023-10-16T08:39:26.401855+0000","last_undegraded":"2023-10-16T08:39:26.401855+0000","last_fullsized":"2023-10-16T08:39:26.401855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:29.539285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951818+0000","last_change":"2023-10-16T08:38:44.662816+0000","last_active":"2023-10-16T08:39:25.951818+0000","last_peered":"2023-10-16T08:39:25.951818+0000","last_clean":"2023-10-16T08:39:25.951818+0000","last_became_active":"2023-10-16T08:38:44.662110+0000","last_became_peered":"2023-10-16T08:38:44.662110+0000","last_unstale":"2023-10-16T08:39:25.951818+0000","last_undegraded":"2023-10-16T08:39:25.951818+0000","last_fullsized":"2023-10-16T08:39:25.951818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:00.201724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955811+0000","last_change":"2023-10-16T08:38:45.662166+0000","last_active":"2023-10-16T08:39:25.955811+0000","last_peered":"2023-10-16T08:39:25.955811+0000","last_clean":"2023-10-16T08:39:25.955811+0000","last_became_active":"2023-10-16T08:38:45.661595+0000","last_became_peered":"2023-10-16T08:38:45.661595+0000","last_unstale":"2023-10-16T08:39:25.955811+0000","last_undegraded":"2023-10-16T08:39:25.955811+0000","last_fullsized":"2023-10-16T08:39:25.955811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:02:39.840175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949503+0000","last_change":"2023-10-16T08:38:44.657766+0000","last_active":"2023-10-16T08:39:25.949503+0000","last_peered":"2023-10-16T08:39:25.949503+0000","last_clean":"2023-10-16T08:39:25.949503+0000","last_became_active":"2023-10-16T08:38:44.657153+0000","last_became_peered":"2023-10-16T08:38:44.657153+0000","last_unstale":"2023-10-16T08:39:25.949503+0000","last_undegraded":"2023-10-16T08:39:25.949503+0000","last_fullsized":"2023-10-16T08:39:25.949503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949501+0000","last_change":"2023-10-16T08:38:45.637591+0000","last_active":"2023-10-16T08:39:25.949501+0000","last_peered":"2023-10-16T08:39:25.949501+0000","last_clean":"2023-10-16T08:39:25.949501+0000","last_became_active":"2023-10-16T08:38:45.635633+0000","last_became_peered":"2023-10-16T08:38:45.635633+0000","last_unstale":"2023-10-16T08:39:25.949501+0000","last_undegraded":"2023-10-16T08:39:25.949501+0000","last_fullsized":"2023-10-16T08:39:25.949501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.946160+0000","last_change":"2023-10-16T08:38:44.638297+0000","last_active":"2023-10-16T08:39:25.946160+0000","last_peered":"2023-10-16T08:39:25.946160+0000","last_clean":"2023-10-16T08:39:25.946160+0000","last_became_active":"2023-10-16T08:38:44.637640+0000","last_became_peered":"2023-10-16T08:38:44.637640+0000","last_unstale":"2023-10-16T08:39:25.946160+0000","last_undegraded":"2023-10-16T08:39:25.946160+0000","last_fullsized":"2023-10-16T08:39:25.946160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:16.003943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954941+0000","last_change":"2023-10-16T08:38:45.657286+0000","last_active":"2023-10-16T08:39:25.954941+0000","last_peered":"2023-10-16T08:39:25.954941+0000","last_clean":"2023-10-16T08:39:25.954941+0000","last_became_active":"2023-10-16T08:38:45.656651+0000","last_became_peered":"2023-10-16T08:38:45.656651+0000","last_unstale":"2023-10-16T08:39:25.954941+0000","last_undegraded":"2023-10-16T08:39:25.954941+0000","last_fullsized":"2023-10-16T08:39:25.954941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:28.052483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403973+0000","last_change":"2023-10-16T08:38:44.648434+0000","last_active":"2023-10-16T08:39:26.403973+0000","last_peered":"2023-10-16T08:39:26.403973+0000","last_clean":"2023-10-16T08:39:26.403973+0000","last_became_active":"2023-10-16T08:38:44.647887+0000","last_became_peered":"2023-10-16T08:38:44.647887+0000","last_unstale":"2023-10-16T08:39:26.403973+0000","last_undegraded":"2023-10-16T08:39:26.403973+0000","last_fullsized":"2023-10-16T08:39:26.403973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954988+0000","last_change":"2023-10-16T08:38:45.661965+0000","last_active":"2023-10-16T08:39:25.954988+0000","last_peered":"2023-10-16T08:39:25.954988+0000","last_clean":"2023-10-16T08:39:25.954988+0000","last_became_active":"2023-10-16T08:38:45.656750+0000","last_became_peered":"2023-10-16T08:38:45.656750+0000","last_unstale":"2023-10-16T08:39:25.954988+0000","last_undegraded":"2023-10-16T08:39:25.954988+0000","last_fullsized":"2023-10-16T08:39:25.954988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945809+0000","last_change":"2023-10-16T08:38:44.647524+0000","last_active":"2023-10-16T08:39:25.945809+0000","last_peered":"2023-10-16T08:39:25.945809+0000","last_clean":"2023-10-16T08:39:25.945809+0000","last_became_active":"2023-10-16T08:38:44.646042+0000","last_became_peered":"2023-10-16T08:38:44.646042+0000","last_unstale":"2023-10-16T08:39:25.945809+0000","last_undegraded":"2023-10-16T08:39:25.945809+0000","last_fullsized":"2023-10-16T08:39:25.945809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:24.375366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955916+0000","last_change":"2023-10-16T08:38:45.670969+0000","last_active":"2023-10-16T08:39:25.955916+0000","last_peered":"2023-10-16T08:39:25.955916+0000","last_clean":"2023-10-16T08:39:25.955916+0000","last_became_active":"2023-10-16T08:38:45.670415+0000","last_became_peered":"2023-10-16T08:38:45.670415+0000","last_unstale":"2023-10-16T08:39:25.955916+0000","last_undegraded":"2023-10-16T08:39:25.955916+0000","last_fullsized":"2023-10-16T08:39:25.955916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:15:00.472153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945149+0000","last_change":"2023-10-16T08:38:44.655744+0000","last_active":"2023-10-16T08:39:25.945149+0000","last_peered":"2023-10-16T08:39:25.945149+0000","last_clean":"2023-10-16T08:39:25.945149+0000","last_became_active":"2023-10-16T08:38:44.655260+0000","last_became_peered":"2023-10-16T08:38:44.655260+0000","last_unstale":"2023-10-16T08:39:25.945149+0000","last_undegraded":"2023-10-16T08:39:25.945149+0000","last_fullsized":"2023-10-16T08:39:25.945149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:54.848445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951801+0000","last_change":"2023-10-16T08:38:45.668230+0000","last_active":"2023-10-16T08:39:25.951801+0000","last_peered":"2023-10-16T08:39:25.951801+0000","last_clean":"2023-10-16T08:39:25.951801+0000","last_became_active":"2023-10-16T08:38:45.667627+0000","last_became_peered":"2023-10-16T08:38:45.667627+0000","last_unstale":"2023-10-16T08:39:25.951801+0000","last_undegraded":"2023-10-16T08:39:25.951801+0000","last_fullsized":"2023-10-16T08:39:25.951801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:28.052483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945644+0000","last_change":"2023-10-16T08:38:44.662118+0000","last_active":"2023-10-16T08:39:25.945644+0000","last_peered":"2023-10-16T08:39:25.945644+0000","last_clean":"2023-10-16T08:39:25.945644+0000","last_became_active":"2023-10-16T08:38:44.661736+0000","last_became_peered":"2023-10-16T08:38:44.661736+0000","last_unstale":"2023-10-16T08:39:25.945644+0000","last_undegraded":"2023-10-16T08:39:25.945644+0000","last_fullsized":"2023-10-16T08:39:25.945644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:49:40.967349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955744+0000","last_change":"2023-10-16T08:38:45.667423+0000","last_active":"2023-10-16T08:39:25.955744+0000","last_peered":"2023-10-16T08:39:25.955744+0000","last_clean":"2023-10-16T08:39:25.955744+0000","last_became_active":"2023-10-16T08:38:45.666390+0000","last_became_peered":"2023-10-16T08:38:45.666390+0000","last_unstale":"2023-10-16T08:39:25.955744+0000","last_undegraded":"2023-10-16T08:39:25.955744+0000","last_fullsized":"2023-10-16T08:39:25.955744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:33.190218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":167,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954612+0000","last_change":"2023-10-16T08:38:34.313511+0000","last_active":"2023-10-16T08:39:25.954612+0000","last_peered":"2023-10-16T08:39:25.954612+0000","last_clean":"2023-10-16T08:39:25.954612+0000","last_became_active":"2023-10-16T08:38:34.310944+0000","last_became_peered":"2023-10-16T08:38:34.310944+0000","last_unstale":"2023-10-16T08:39:25.954612+0000","last_undegraded":"2023-10-16T08:39:25.954612+0000","last_fullsized":"2023-10-16T08:39:25.954612+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:33.264692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:33.264692+0000","last_clean_scrub_stamp":"2023-10-16T08:38:33.264692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:50.170461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951158+0000","last_change":"2023-10-16T08:38:44.655223+0000","last_active":"2023-10-16T08:39:25.951158+0000","last_peered":"2023-10-16T08:39:25.951158+0000","last_clean":"2023-10-16T08:39:25.951158+0000","last_became_active":"2023-10-16T08:38:44.654277+0000","last_became_peered":"2023-10-16T08:38:44.654277+0000","last_unstale":"2023-10-16T08:39:25.951158+0000","last_undegraded":"2023-10-16T08:39:25.951158+0000","last_fullsized":"2023-10-16T08:39:25.951158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:03.462433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955641+0000","last_change":"2023-10-16T08:38:45.679868+0000","last_active":"2023-10-16T08:39:25.955641+0000","last_peered":"2023-10-16T08:39:25.955641+0000","last_clean":"2023-10-16T08:39:25.955641+0000","last_became_active":"2023-10-16T08:38:45.679409+0000","last_became_peered":"2023-10-16T08:38:45.679409+0000","last_unstale":"2023-10-16T08:39:25.955641+0000","last_undegraded":"2023-10-16T08:39:25.955641+0000","last_fullsized":"2023-10-16T08:39:25.955641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954528+0000","last_change":"2023-10-16T08:38:44.652152+0000","last_active":"2023-10-16T08:39:25.954528+0000","last_peered":"2023-10-16T08:39:25.954528+0000","last_clean":"2023-10-16T08:39:25.954528+0000","last_became_active":"2023-10-16T08:38:44.651534+0000","last_became_peered":"2023-10-16T08:38:44.651534+0000","last_unstale":"2023-10-16T08:39:25.954528+0000","last_undegraded":"2023-10-16T08:39:25.954528+0000","last_fullsized":"2023-10-16T08:39:25.954528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.401719+0000","last_change":"2023-10-16T08:38:45.651786+0000","last_active":"2023-10-16T08:39:26.401719+0000","last_peered":"2023-10-16T08:39:26.401719+0000","last_clean":"2023-10-16T08:39:26.401719+0000","last_became_active":"2023-10-16T08:38:45.650715+0000","last_became_peered":"2023-10-16T08:38:45.650715+0000","last_unstale":"2023-10-16T08:39:26.401719+0000","last_undegraded":"2023-10-16T08:39:26.401719+0000","last_fullsized":"2023-10-16T08:39:26.401719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:29.799799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.946109+0000","last_change":"2023-10-16T08:38:44.645933+0000","last_active":"2023-10-16T08:39:25.946109+0000","last_peered":"2023-10-16T08:39:25.946109+0000","last_clean":"2023-10-16T08:39:25.946109+0000","last_became_active":"2023-10-16T08:38:44.645451+0000","last_became_peered":"2023-10-16T08:38:44.645451+0000","last_unstale":"2023-10-16T08:39:25.946109+0000","last_undegraded":"2023-10-16T08:39:25.946109+0000","last_fullsized":"2023-10-16T08:39:25.946109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.944780+0000","last_change":"2023-10-16T08:38:45.641499+0000","last_active":"2023-10-16T08:39:25.944780+0000","last_peered":"2023-10-16T08:39:25.944780+0000","last_clean":"2023-10-16T08:39:25.944780+0000","last_became_active":"2023-10-16T08:38:45.640765+0000","last_became_peered":"2023-10-16T08:38:45.640765+0000","last_unstale":"2023-10-16T08:39:25.944780+0000","last_undegraded":"2023-10-16T08:39:25.944780+0000","last_fullsized":"2023-10-16T08:39:25.944780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.952050+0000","last_change":"2023-10-16T08:38:44.655401+0000","last_active":"2023-10-16T08:39:25.952050+0000","last_peered":"2023-10-16T08:39:25.952050+0000","last_clean":"2023-10-16T08:39:25.952050+0000","last_became_active":"2023-10-16T08:38:44.654622+0000","last_became_peered":"2023-10-16T08:38:44.654622+0000","last_unstale":"2023-10-16T08:39:25.952050+0000","last_undegraded":"2023-10-16T08:39:25.952050+0000","last_fullsized":"2023-10-16T08:39:25.952050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:48.705915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403580+0000","last_change":"2023-10-16T08:38:45.659377+0000","last_active":"2023-10-16T08:39:26.403580+0000","last_peered":"2023-10-16T08:39:26.403580+0000","last_clean":"2023-10-16T08:39:26.403580+0000","last_became_active":"2023-10-16T08:38:45.658594+0000","last_became_peered":"2023-10-16T08:38:45.658594+0000","last_unstale":"2023-10-16T08:39:26.403580+0000","last_undegraded":"2023-10-16T08:39:26.403580+0000","last_fullsized":"2023-10-16T08:39:26.403580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:56.738058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945087+0000","last_change":"2023-10-16T08:38:44.634158+0000","last_active":"2023-10-16T08:39:25.945087+0000","last_peered":"2023-10-16T08:39:25.945087+0000","last_clean":"2023-10-16T08:39:25.945087+0000","last_became_active":"2023-10-16T08:38:44.633854+0000","last_became_peered":"2023-10-16T08:38:44.633854+0000","last_unstale":"2023-10-16T08:39:25.945087+0000","last_undegraded":"2023-10-16T08:39:25.945087+0000","last_fullsized":"2023-10-16T08:39:25.945087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:55.727203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.015974+0000","last_change":"2023-10-16T08:39:27.015974+0000","last_active":"2023-10-16T08:39:27.015974+0000","last_peered":"2023-10-16T08:39:27.015974+0000","last_clean":"2023-10-16T08:39:27.015974+0000","last_became_active":"2023-10-16T08:39:27.015266+0000","last_became_peered":"2023-10-16T08:39:27.015266+0000","last_unstale":"2023-10-16T08:39:27.015974+0000","last_undegraded":"2023-10-16T08:39:27.015974+0000","last_fullsized":"2023-10-16T08:39:27.015974+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:41:52.951028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956207+0000","last_change":"2023-10-16T08:38:44.655693+0000","last_active":"2023-10-16T08:39:25.956207+0000","last_peered":"2023-10-16T08:39:25.956207+0000","last_clean":"2023-10-16T08:39:25.956207+0000","last_became_active":"2023-10-16T08:38:44.655001+0000","last_became_peered":"2023-10-16T08:38:44.655001+0000","last_unstale":"2023-10-16T08:39:25.956207+0000","last_undegraded":"2023-10-16T08:39:25.956207+0000","last_fullsized":"2023-10-16T08:39:25.956207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:30.649639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945004+0000","last_change":"2023-10-16T08:38:45.639658+0000","last_active":"2023-10-16T08:39:25.945004+0000","last_peered":"2023-10-16T08:39:25.945004+0000","last_clean":"2023-10-16T08:39:25.945004+0000","last_became_active":"2023-10-16T08:38:45.638702+0000","last_became_peered":"2023-10-16T08:38:45.638702+0000","last_unstale":"2023-10-16T08:39:25.945004+0000","last_undegraded":"2023-10-16T08:39:25.945004+0000","last_fullsized":"2023-10-16T08:39:25.945004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"20'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945364+0000","last_change":"2023-10-16T08:38:44.648715+0000","last_active":"2023-10-16T08:39:25.945364+0000","last_peered":"2023-10-16T08:39:25.945364+0000","last_clean":"2023-10-16T08:39:25.945364+0000","last_became_active":"2023-10-16T08:38:44.648312+0000","last_became_peered":"2023-10-16T08:38:44.648312+0000","last_unstale":"2023-10-16T08:39:25.945364+0000","last_undegraded":"2023-10-16T08:39:25.945364+0000","last_fullsized":"2023-10-16T08:39:25.945364+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955768+0000","last_change":"2023-10-16T08:38:45.663402+0000","last_active":"2023-10-16T08:39:25.955768+0000","last_peered":"2023-10-16T08:39:25.955768+0000","last_clean":"2023-10-16T08:39:25.955768+0000","last_became_active":"2023-10-16T08:38:45.662442+0000","last_became_peered":"2023-10-16T08:38:45.662442+0000","last_unstale":"2023-10-16T08:39:25.955768+0000","last_undegraded":"2023-10-16T08:39:25.955768+0000","last_fullsized":"2023-10-16T08:39:25.955768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949981+0000","last_change":"2023-10-16T08:38:44.650015+0000","last_active":"2023-10-16T08:39:25.949981+0000","last_peered":"2023-10-16T08:39:25.949981+0000","last_clean":"2023-10-16T08:39:25.949981+0000","last_became_active":"2023-10-16T08:38:44.648155+0000","last_became_peered":"2023-10-16T08:38:44.648155+0000","last_unstale":"2023-10-16T08:39:25.949981+0000","last_undegraded":"2023-10-16T08:39:25.949981+0000","last_fullsized":"2023-10-16T08:39:25.949981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402266+0000","last_change":"2023-10-16T08:38:45.637568+0000","last_active":"2023-10-16T08:39:26.402266+0000","last_peered":"2023-10-16T08:39:26.402266+0000","last_clean":"2023-10-16T08:39:26.402266+0000","last_became_active":"2023-10-16T08:38:45.636751+0000","last_became_peered":"2023-10-16T08:38:45.636751+0000","last_unstale":"2023-10-16T08:39:26.402266+0000","last_undegraded":"2023-10-16T08:39:26.402266+0000","last_fullsized":"2023-10-16T08:39:26.402266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954444+0000","last_change":"2023-10-16T08:38:44.649257+0000","last_active":"2023-10-16T08:39:25.954444+0000","last_peered":"2023-10-16T08:39:25.954444+0000","last_clean":"2023-10-16T08:39:25.954444+0000","last_became_active":"2023-10-16T08:38:44.646990+0000","last_became_peered":"2023-10-16T08:38:44.646990+0000","last_unstale":"2023-10-16T08:39:25.954444+0000","last_undegraded":"2023-10-16T08:39:25.954444+0000","last_fullsized":"2023-10-16T08:39:25.954444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403442+0000","last_change":"2023-10-16T08:38:45.648228+0000","last_active":"2023-10-16T08:39:26.403442+0000","last_peered":"2023-10-16T08:39:26.403442+0000","last_clean":"2023-10-16T08:39:26.403442+0000","last_became_active":"2023-10-16T08:38:45.647449+0000","last_became_peered":"2023-10-16T08:38:45.647449+0000","last_unstale":"2023-10-16T08:39:26.403442+0000","last_undegraded":"2023-10-16T08:39:26.403442+0000","last_fullsized":"2023-10-16T08:39:26.403442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:20.779617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.944878+0000","last_change":"2023-10-16T08:38:44.631354+0000","last_active":"2023-10-16T08:39:25.944878+0000","last_peered":"2023-10-16T08:39:25.944878+0000","last_clean":"2023-10-16T08:39:25.944878+0000","last_became_active":"2023-10-16T08:38:44.630451+0000","last_became_peered":"2023-10-16T08:38:44.630451+0000","last_unstale":"2023-10-16T08:39:25.944878+0000","last_undegraded":"2023-10-16T08:39:25.944878+0000","last_fullsized":"2023-10-16T08:39:25.944878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:10.980982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955719+0000","last_change":"2023-10-16T08:38:45.663898+0000","last_active":"2023-10-16T08:39:25.955719+0000","last_peered":"2023-10-16T08:39:25.955719+0000","last_clean":"2023-10-16T08:39:25.955719+0000","last_became_active":"2023-10-16T08:38:45.663100+0000","last_became_peered":"2023-10-16T08:38:45.663100+0000","last_unstale":"2023-10-16T08:39:25.955719+0000","last_undegraded":"2023-10-16T08:39:25.955719+0000","last_fullsized":"2023-10-16T08:39:25.955719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945128+0000","last_change":"2023-10-16T08:38:44.646987+0000","last_active":"2023-10-16T08:39:25.945128+0000","last_peered":"2023-10-16T08:39:25.945128+0000","last_clean":"2023-10-16T08:39:25.945128+0000","last_became_active":"2023-10-16T08:38:44.646580+0000","last_became_peered":"2023-10-16T08:38:44.646580+0000","last_unstale":"2023-10-16T08:39:25.945128+0000","last_undegraded":"2023-10-16T08:39:25.945128+0000","last_fullsized":"2023-10-16T08:39:25.945128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956260+0000","last_change":"2023-10-16T08:38:45.662894+0000","last_active":"2023-10-16T08:39:25.956260+0000","last_peered":"2023-10-16T08:39:25.956260+0000","last_clean":"2023-10-16T08:39:25.956260+0000","last_became_active":"2023-10-16T08:38:45.662226+0000","last_became_peered":"2023-10-16T08:38:45.662226+0000","last_unstale":"2023-10-16T08:39:25.956260+0000","last_undegraded":"2023-10-16T08:39:25.956260+0000","last_fullsized":"2023-10-16T08:39:25.956260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955552+0000","last_change":"2023-10-16T08:38:44.660571+0000","last_active":"2023-10-16T08:39:25.955552+0000","last_peered":"2023-10-16T08:39:25.955552+0000","last_clean":"2023-10-16T08:39:25.955552+0000","last_became_active":"2023-10-16T08:38:44.659976+0000","last_became_peered":"2023-10-16T08:38:44.659976+0000","last_unstale":"2023-10-16T08:39:25.955552+0000","last_undegraded":"2023-10-16T08:39:25.955552+0000","last_fullsized":"2023-10-16T08:39:25.955552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949392+0000","last_change":"2023-10-16T08:38:45.665200+0000","last_active":"2023-10-16T08:39:25.949392+0000","last_peered":"2023-10-16T08:39:25.949392+0000","last_clean":"2023-10-16T08:39:25.949392+0000","last_became_active":"2023-10-16T08:38:45.664616+0000","last_became_peered":"2023-10-16T08:38:45.664616+0000","last_unstale":"2023-10-16T08:39:25.949392+0000","last_undegraded":"2023-10-16T08:39:25.949392+0000","last_fullsized":"2023-10-16T08:39:25.949392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.953546+0000","last_change":"2023-10-16T08:38:45.680892+0000","last_active":"2023-10-16T08:39:25.953546+0000","last_peered":"2023-10-16T08:39:25.953546+0000","last_clean":"2023-10-16T08:39:25.953546+0000","last_became_active":"2023-10-16T08:38:45.680231+0000","last_became_peered":"2023-10-16T08:38:45.680231+0000","last_unstale":"2023-10-16T08:39:25.953546+0000","last_undegraded":"2023-10-16T08:39:25.953546+0000","last_fullsized":"2023-10-16T08:39:25.953546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:58:51.599709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956015+0000","last_change":"2023-10-16T08:38:44.664269+0000","last_active":"2023-10-16T08:39:25.956015+0000","last_peered":"2023-10-16T08:39:25.956015+0000","last_clean":"2023-10-16T08:39:25.956015+0000","last_became_active":"2023-10-16T08:38:44.663674+0000","last_became_peered":"2023-10-16T08:38:44.663674+0000","last_unstale":"2023-10-16T08:39:25.956015+0000","last_undegraded":"2023-10-16T08:39:25.956015+0000","last_fullsized":"2023-10-16T08:39:25.956015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951695+0000","last_change":"2023-10-16T08:38:45.680890+0000","last_active":"2023-10-16T08:39:25.951695+0000","last_peered":"2023-10-16T08:39:25.951695+0000","last_clean":"2023-10-16T08:39:25.951695+0000","last_became_active":"2023-10-16T08:38:45.680266+0000","last_became_peered":"2023-10-16T08:38:45.680266+0000","last_unstale":"2023-10-16T08:39:25.951695+0000","last_undegraded":"2023-10-16T08:39:25.951695+0000","last_fullsized":"2023-10-16T08:39:25.951695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:26.292137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955778+0000","last_change":"2023-10-16T08:38:44.653447+0000","last_active":"2023-10-16T08:39:25.955778+0000","last_peered":"2023-10-16T08:39:25.955778+0000","last_clean":"2023-10-16T08:39:25.955778+0000","last_became_active":"2023-10-16T08:38:44.652658+0000","last_became_peered":"2023-10-16T08:38:44.652658+0000","last_unstale":"2023-10-16T08:39:25.955778+0000","last_undegraded":"2023-10-16T08:39:25.955778+0000","last_fullsized":"2023-10-16T08:39:25.955778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:22.350066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402179+0000","last_change":"2023-10-16T08:38:45.665041+0000","last_active":"2023-10-16T08:39:26.402179+0000","last_peered":"2023-10-16T08:39:26.402179+0000","last_clean":"2023-10-16T08:39:26.402179+0000","last_became_active":"2023-10-16T08:38:45.664274+0000","last_became_peered":"2023-10-16T08:38:45.664274+0000","last_unstale":"2023-10-16T08:39:26.402179+0000","last_undegraded":"2023-10-16T08:39:26.402179+0000","last_fullsized":"2023-10-16T08:39:26.402179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955588+0000","last_change":"2023-10-16T08:38:44.645754+0000","last_active":"2023-10-16T08:39:25.955588+0000","last_peered":"2023-10-16T08:39:25.955588+0000","last_clean":"2023-10-16T08:39:25.955588+0000","last_became_active":"2023-10-16T08:38:44.645000+0000","last_became_peered":"2023-10-16T08:38:44.645000+0000","last_unstale":"2023-10-16T08:39:25.955588+0000","last_undegraded":"2023-10-16T08:39:25.955588+0000","last_fullsized":"2023-10-16T08:39:25.955588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955495+0000","last_change":"2023-10-16T08:38:45.664252+0000","last_active":"2023-10-16T08:39:25.955495+0000","last_peered":"2023-10-16T08:39:25.955495+0000","last_clean":"2023-10-16T08:39:25.955495+0000","last_became_active":"2023-10-16T08:38:45.663538+0000","last_became_peered":"2023-10-16T08:38:45.663538+0000","last_unstale":"2023-10-16T08:39:25.955495+0000","last_undegraded":"2023-10-16T08:39:25.955495+0000","last_fullsized":"2023-10-16T08:39:25.955495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402379+0000","last_change":"2023-10-16T08:38:44.646321+0000","last_active":"2023-10-16T08:39:26.402379+0000","last_peered":"2023-10-16T08:39:26.402379+0000","last_clean":"2023-10-16T08:39:26.402379+0000","last_became_active":"2023-10-16T08:38:44.645854+0000","last_became_peered":"2023-10-16T08:38:44.645854+0000","last_unstale":"2023-10-16T08:39:26.402379+0000","last_undegraded":"2023-10-16T08:39:26.402379+0000","last_fullsized":"2023-10-16T08:39:26.402379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:10.980982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6065,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":59,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":12130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":827392,"data_stored":674262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":1.3220000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.1220000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.97699999999999998}]},{"osd":2,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":1.228}]},{"osd":3,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":1.363}]},{"osd":4,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":1.0569999999999999}]},{"osd":5,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.276},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":1.0900000000000001}]},{"osd":6,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":1.01}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":88579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.57499999999999996},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":0.63600000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.83699999999999997},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":0.70599999999999996}]},{"osd":2,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.98399999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.94099999999999995}]},{"osd":3,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":1.1140000000000001},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":1.1930000000000001}]},{"osd":4,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.88500000000000001},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":1.25}]},{"osd":5,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":1.1519999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.3600000000000001}]},{"osd":7,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.04,"5min":2.04,"15min":2.04},"last":1.4430000000000001},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.3029999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574866,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":83402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":1.0509999999999999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.87}]},{"osd":1,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.165},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":1.0269999999999999}]},{"osd":2,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.133}]},{"osd":3,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.087},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.198}]},{"osd":4,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.98899999999999999}]},{"osd":6,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.91000000000000003},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":0.84399999999999997}]},{"osd":7,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":1.2230000000000001},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.111}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":83438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.48799999999999999}]},{"osd":1,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.004}]},{"osd":2,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.96799999999999997},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.93799999999999994}]},{"osd":3,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.67000000000000004},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.873}]},{"osd":5,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":1.05},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":0.754}]},{"osd":6,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.71499999999999997},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":1.1060000000000001}]},{"osd":7,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.83399999999999996},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.153,"5min":2.153,"15min":2.153},"last":1.079}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":88579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.51600000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.64400000000000002}]},{"osd":2,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.73099999999999998}]},{"osd":4,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.77400000000000002}]},{"osd":5,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.67600000000000005},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.82699999999999996}]},{"osd":6,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.95499999999999996},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":0.85999999999999999}]},{"osd":7,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.0349999999999999},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":0.995}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":83312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.96699999999999997},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.64700000000000002}]},{"osd":1,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.752}]},{"osd":3,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.79000000000000004}]},{"osd":4,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.119},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.93999999999999995}]},{"osd":5,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.90500000000000003},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.23}]},{"osd":6,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.056},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":1.0840000000000001}]},{"osd":7,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":1.1679999999999999},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.1970000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574864,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":83472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.48399999999999999}]},{"osd":2,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.68899999999999995}]},{"osd":3,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.60499999999999998}]},{"osd":4,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.70599999999999996}]},{"osd":5,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.123}]},{"osd":6,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":1.0049999999999999}]},{"osd":7,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.046},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.5449999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":0.94999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":835584,"data_stored":674318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.32000000000000001},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.61499999999999999}]},{"osd":2,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.72199999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.56200000000000006}]},{"osd":4,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.75900000000000001}]},{"osd":5,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.81299999999999994},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.97599999999999998}]},{"osd":6,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":0.91700000000000004}]},{"osd":7,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":1.0569999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:39:43.269 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:39:43.290 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:39:43.306 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:39:43.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.419+0000 7f57e7104640 1 Processor -- start 2023-10-16T08:39:43.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.419+0000 7f57e7104640 1 -- start start 2023-10-16T08:39:43.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57e7104640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57e00cae50 0x7f57e006b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:43.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57e7104640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e00adca0 con 0x7f57e015b040 2023-10-16T08:39:43.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57e7104640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57e00c5840 con 0x7f57e00adec0 2023-10-16T08:39:43.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57e7104640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00cf890 con 0x7f57e00cae50 2023-10-16T08:39:43.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57e4ea0640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f57e015b040 0x7f57e00ca740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:45630/0 (socket says 172.21.15.40:45630) 2023-10-16T08:39:43.432 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57e4ea0640 1 -- 172.21.15.40:0/4086705326 learned_addr learned my addr 172.21.15.40:0/4086705326 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:43.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57dffff640 1 --2- 172.21.15.40:0/4086705326 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57e00cae50 0x7f57e006b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:43.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57df7fe640 1 -- 172.21.15.40:0/4086705326 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2472849959 0 0) 0x7f57e00adca0 con 0x7f57e015b040 2023-10-16T08:39:43.433 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57df7fe640 1 -- 172.21.15.40:0/4086705326 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57cc003530 con 0x7f57e015b040 2023-10-16T08:39:43.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57df7fe640 1 -- 172.21.15.40:0/4086705326 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3565886051 0 0) 0x7f57e00c5840 con 0x7f57e00adec0 2023-10-16T08:39:43.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57df7fe640 1 -- 172.21.15.40:0/4086705326 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57e00adca0 con 0x7f57e00adec0 2023-10-16T08:39:43.434 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57df7fe640 1 -- 172.21.15.40:0/4086705326 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3204627263 0 0) 0x7f57cc003530 con 0x7f57e015b040 2023-10-16T08:39:43.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57df7fe640 1 -- 172.21.15.40:0/4086705326 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57e00cae50 msgr2=0x7f57e006b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:43.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57df7fe640 1 --2- 172.21.15.40:0/4086705326 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57e00cae50 0x7f57e006b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:43.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57df7fe640 1 -- 172.21.15.40:0/4086705326 >> v1:172.21.15.86:6789/0 conn(0x7f57e00adec0 legacy=0x7f57e006bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:43.435 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57df7fe640 1 -- 172.21.15.40:0/4086705326 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e00cf5b0 con 0x7f57e015b040 2023-10-16T08:39:43.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57df7fe640 1 -- 172.21.15.40:0/4086705326 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f57c8001680 con 0x7f57e015b040 2023-10-16T08:39:43.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57df7fe640 1 -- 172.21.15.40:0/4086705326 >> v1:172.21.15.40:6789/0 conn(0x7f57e015b040 legacy=0x7f57e00ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:43.436 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57df7fe640 1 --2- 172.21.15.40:0/4086705326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57cc003bb0 0x7f57cc003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:43.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57df7fe640 1 --2- 172.21.15.40:0/4086705326 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57e00cae50 0x7f57cc0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:43.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57df7fe640 1 --2- 172.21.15.40:0/4086705326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57cc004d10 0x7f57cc005170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:43.437 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57df7fe640 1 -- 172.21.15.40:0/4086705326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00cf890 con 0x7f57cc003bb0 2023-10-16T08:39:43.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57df7fe640 1 -- 172.21.15.40:0/4086705326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc00bba0 con 0x7f57cc004d10 2023-10-16T08:39:43.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57df7fe640 1 -- 172.21.15.40:0/4086705326 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc00bd60 con 0x7f57e00cae50 2023-10-16T08:39:43.438 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57e4ea0640 1 --2- 172.21.15.40:0/4086705326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57cc003bb0 0x7f57cc003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:43.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57dffff640 1 --2- 172.21.15.40:0/4086705326 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57e00cae50 0x7f57cc0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:43.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57e56a1640 1 --2- 172.21.15.40:0/4086705326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57cc004d10 0x7f57cc005170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:43.439 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57e4ea0640 1 -- 172.21.15.40:0/4086705326 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57e00cae50 msgr2=0x7f57cc0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:43.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57e4ea0640 1 --2- 172.21.15.40:0/4086705326 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57e00cae50 0x7f57cc0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:43.440 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57e4ea0640 1 -- 172.21.15.40:0/4086705326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57cc004d10 msgr2=0x7f57cc005170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:43.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57e4ea0640 1 --2- 172.21.15.40:0/4086705326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57cc004d10 0x7f57cc005170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:43.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57e4ea0640 1 -- 172.21.15.40:0/4086705326 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57cc003530 con 0x7f57cc003bb0 2023-10-16T08:39:43.441 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.423+0000 7f57dffff640 1 --2- 172.21.15.40:0/4086705326 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57e00cae50 0x7f57cc0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:43.445 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.427+0000 7f57e4ea0640 1 --2- 172.21.15.40:0/4086705326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57cc003bb0 0x7f57cc003fb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f57c800c160 tx=0x7f57e006cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7280d4a0860d5597 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:43.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.427+0000 7f57df7fe640 1 -- 172.21.15.40:0/4086705326 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c80058e0 con 0x7f57cc003bb0 2023-10-16T08:39:43.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.427+0000 7f57df7fe640 1 -- 172.21.15.40:0/4086705326 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57c8005a60 con 0x7f57cc003bb0 2023-10-16T08:39:43.446 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.427+0000 7f57df7fe640 1 -- 172.21.15.40:0/4086705326 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c8005e10 con 0x7f57cc003bb0 2023-10-16T08:39:43.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.427+0000 7f57e7104640 1 -- 172.21.15.40:0/4086705326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57cc003bb0 msgr2=0x7f57cc003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:43.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.427+0000 7f57e7104640 1 --2- 172.21.15.40:0/4086705326 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57cc003bb0 0x7f57cc003fb0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f57c800c160 tx=0x7f57e006cc10 comp rx=0 tx=0).stop 2023-10-16T08:39:43.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.427+0000 7f57e56a1640 1 -- 172.21.15.40:0/4086705326 reap_dead start 2023-10-16T08:39:43.447 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.427+0000 7f57e7104640 1 -- 172.21.15.40:0/4086705326 shutdown_connections 2023-10-16T08:39:43.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.427+0000 7f57e7104640 1 -- 172.21.15.40:0/4086705326 >> 172.21.15.40:0/4086705326 conn(0x7f57e00ce960 msgr2=0x7f57e00ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:43.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.427+0000 7f57e7104640 1 -- 172.21.15.40:0/4086705326 shutdown_connections 2023-10-16T08:39:43.448 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.427+0000 7f57e7104640 1 -- 172.21.15.40:0/4086705326 wait complete. 2023-10-16T08:39:43.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.427+0000 7f57e7104640 1 Processor -- start 2023-10-16T08:39:43.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.427+0000 7f57e7104640 1 -- start start 2023-10-16T08:39:43.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57e7104640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57cc003bb0 0x7f57e01f6810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:43.449 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57e7104640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e0161300 0x7f57e00c6b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:43.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57e7104640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57e00c7a00 0x7f57e00af500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:43.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57e7104640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e01f7740 con 0x7f57e0161300 2023-10-16T08:39:43.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57e7104640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57e00c70b0 con 0x7f57cc003bb0 2023-10-16T08:39:43.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57e7104640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f57e0161780 con 0x7f57e00c7a00 2023-10-16T08:39:43.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57dffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e0161300 0x7f57e00c6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:43.451 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57dffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e0161300 0x7f57e00c6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:50748/0 (socket says 172.21.15.40:50748) 2023-10-16T08:39:43.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57dffff640 1 -- 172.21.15.40:0/2518827064 learned_addr learned my addr 172.21.15.40:0/2518827064 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:43.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57e4ea0640 1 --2- 172.21.15.40:0/2518827064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57cc003bb0 0x7f57e01f6810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:43.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57e56a1640 1 --2- 172.21.15.40:0/2518827064 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57e00c7a00 0x7f57e00af500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:43.452 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57dffff640 1 -- 172.21.15.40:0/2518827064 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57e00c7a00 msgr2=0x7f57e00af500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:43.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57dffff640 1 --2- 172.21.15.40:0/2518827064 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57e00c7a00 0x7f57e00af500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:43.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57dffff640 1 -- 172.21.15.40:0/2518827064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57cc003bb0 msgr2=0x7f57e01f6810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:43.453 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57dffff640 1 --2- 172.21.15.40:0/2518827064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57cc003bb0 0x7f57e01f6810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:43.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57dffff640 1 -- 172.21.15.40:0/2518827064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57e01f6ea0 con 0x7f57e0161300 2023-10-16T08:39:43.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57dffff640 1 --2- 172.21.15.40:0/2518827064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e0161300 0x7f57e00c6b70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f57d00116a0 tx=0x7f57d0011b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:43.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57dd7fa640 1 -- 172.21.15.40:0/2518827064 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d0004220 con 0x7f57e0161300 2023-10-16T08:39:43.454 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57dd7fa640 1 -- 172.21.15.40:0/2518827064 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57d0004380 con 0x7f57e0161300 2023-10-16T08:39:43.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57dd7fa640 1 -- 172.21.15.40:0/2518827064 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57d000f9e0 con 0x7f57e0161300 2023-10-16T08:39:43.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57e7104640 1 -- 172.21.15.40:0/2518827064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57e01f7180 con 0x7f57e0161300 2023-10-16T08:39:43.455 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57e7104640 1 -- 172.21.15.40:0/2518827064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57e0155260 con 0x7f57e0161300 2023-10-16T08:39:43.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57e7104640 1 -- 172.21.15.40:0/2518827064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57e004e9c0 con 0x7f57e0161300 2023-10-16T08:39:43.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57dd7fa640 1 -- 172.21.15.40:0/2518827064 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f57d000fb40 con 0x7f57e0161300 2023-10-16T08:39:43.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57dd7fa640 1 --2- 172.21.15.40:0/2518827064 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f57c00664a0 0x7f57c0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:43.456 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57dd7fa640 1 -- 172.21.15.40:0/2518827064 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f57d0084440 con 0x7f57e0161300 2023-10-16T08:39:43.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.431+0000 7f57e4ea0640 1 --2- 172.21.15.40:0/2518827064 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f57c00664a0 0x7f57c0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:43.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.435+0000 7f57e4ea0640 1 --2- 172.21.15.40:0/2518827064 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f57c00664a0 0x7f57c0068960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f57c80aa000 tx=0x7f57c8001990 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:43.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.435+0000 7f57dd7fa640 1 -- 172.21.15.40:0/2518827064 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f57d00518c0 con 0x7f57e0161300 2023-10-16T08:39:43.602 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.595+0000 7f57e7104640 1 -- 172.21.15.40:0/2518827064 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57e006ece0 con 0x7f57e0161300 2023-10-16T08:39:43.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.599+0000 7f57dd7fa640 1 -- 172.21.15.40:0/2518827064 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16741 (secure 0 0 0) 0x7f57d0054ee0 con 0x7f57e0161300 2023-10-16T08:39:43.605 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:39:43.606 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:39:26.948454+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:39:43.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.603+0000 7f57e7104640 1 -- 172.21.15.40:0/2518827064 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f57c00664a0 msgr2=0x7f57c0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.603+0000 7f57e7104640 1 --2- 172.21.15.40:0/2518827064 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f57c00664a0 0x7f57c0068960 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f57c80aa000 tx=0x7f57c8001990 comp rx=0 tx=0).stop 2023-10-16T08:39:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.603+0000 7f57e7104640 1 -- 172.21.15.40:0/2518827064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e0161300 msgr2=0x7f57e00c6b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:43.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.603+0000 7f57e7104640 1 --2- 172.21.15.40:0/2518827064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e0161300 0x7f57e00c6b70 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f57d00116a0 tx=0x7f57d0011b60 comp rx=0 tx=0).stop 2023-10-16T08:39:43.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.603+0000 7f57e7104640 1 -- 172.21.15.40:0/2518827064 shutdown_connections 2023-10-16T08:39:43.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.603+0000 7f57e7104640 1 --2- 172.21.15.40:0/2518827064 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f57e00c7a00 0x7f57e00af500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:43.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.603+0000 7f57e7104640 1 --2- 172.21.15.40:0/2518827064 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f57c00664a0 0x7f57c0068960 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:43.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.603+0000 7f57e7104640 1 --2- 172.21.15.40:0/2518827064 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f57e0161300 0x7f57e00c6b70 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:43.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.603+0000 7f57e7104640 1 --2- 172.21.15.40:0/2518827064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57cc003bb0 0x7f57e01f6810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:43.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.603+0000 7f57e7104640 1 -- 172.21.15.40:0/2518827064 >> 172.21.15.40:0/2518827064 conn(0x7f57e00ce960 msgr2=0x7f57e00ca1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:43.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.603+0000 7f57e7104640 1 -- 172.21.15.40:0/2518827064 shutdown_connections 2023-10-16T08:39:43.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:43.603+0000 7f57e7104640 1 -- 172.21.15.40:0/2518827064 wait complete. 2023-10-16T08:39:44.626 INFO:tasks.ceph:Scrubbing osd.0 2023-10-16T08:39:44.627 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-16T08:39:44.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.759+0000 7f0ac52eb640 1 Processor -- start 2023-10-16T08:39:44.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.759+0000 7f0ac52eb640 1 -- start start 2023-10-16T08:39:44.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.759+0000 7f0ac52eb640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0ac015ab40 0x7f0ac00cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:44.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.759+0000 7f0ac52eb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ac015af20 0x7f0ac0070a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:44.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.759+0000 7f0ac52eb640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ac00ae070 con 0x7f0ac00b41e0 2023-10-16T08:39:44.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.759+0000 7f0ac52eb640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ac006c3c0 con 0x7f0ac015ab40 2023-10-16T08:39:44.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.759+0000 7f0ac52eb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00b4750 con 0x7f0ac015af20 2023-10-16T08:39:44.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.759+0000 7f0abeffd640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f0ac015ab40 0x7f0ac00cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:44.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.759+0000 7f0abf7fe640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f0ac00b41e0 0x7f0ac00ca460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.40:35810/0 (socket says 172.21.15.40:35810) 2023-10-16T08:39:44.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.759+0000 7f0ab67fc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ac015af20 0x7f0ac0070a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:44.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.763+0000 7f0abf7fe640 1 -- 172.21.15.40:0/4002094851 learned_addr learned my addr 172.21.15.40:0/4002094851 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:44.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.763+0000 7f0abe7fc640 1 -- 172.21.15.40:0/4002094851 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3771019786 0 0) 0x7f0ac00ae070 con 0x7f0ac00b41e0 2023-10-16T08:39:44.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.763+0000 7f0abe7fc640 1 -- 172.21.15.40:0/4002094851 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0aa0003530 con 0x7f0ac00b41e0 2023-10-16T08:39:44.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.763+0000 7f0ab67fc640 1 -- 172.21.15.40:0/4002094851 >> v2:172.21.15.86:3300/0 conn(0x7f0ac015ab40 msgr2=0x7f0ac00cf3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:44.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.763+0000 7f0ab67fc640 1 --2- 172.21.15.40:0/4002094851 >> v2:172.21.15.86:3300/0 conn(0x7f0ac015ab40 0x7f0ac00cf3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:44.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.763+0000 7f0ab67fc640 1 -- 172.21.15.40:0/4002094851 >> v1:172.21.15.86:6789/0 conn(0x7f0ac00b41e0 legacy=0x7f0ac00ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:44.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.763+0000 7f0ab67fc640 1 -- 172.21.15.40:0/4002094851 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ac015e890 con 0x7f0ac015af20 2023-10-16T08:39:44.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.767+0000 7f0ab67fc640 1 --2- 172.21.15.40:0/4002094851 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ac015af20 0x7f0ac0070a10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80097c0 tx=0x7f0aa801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=68b3cc524588b958 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:44.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.767+0000 7f0abe7fc640 1 -- 172.21.15.40:0/4002094851 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa80ac620 con 0x7f0ac015af20 2023-10-16T08:39:44.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.767+0000 7f0abe7fc640 1 -- 172.21.15.40:0/4002094851 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa80ac780 con 0x7f0ac015af20 2023-10-16T08:39:44.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.767+0000 7f0abe7fc640 1 -- 172.21.15.40:0/4002094851 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa80acaf0 con 0x7f0ac015af20 2023-10-16T08:39:44.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.767+0000 7f0ac52eb640 1 -- 172.21.15.40:0/4002094851 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ac015af20 msgr2=0x7f0ac0070a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:44.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.767+0000 7f0ac52eb640 1 --2- 172.21.15.40:0/4002094851 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ac015af20 0x7f0ac0070a10 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0aa80097c0 tx=0x7f0aa801c3c0 comp rx=0 tx=0).stop 2023-10-16T08:39:44.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.767+0000 7f0ac52eb640 1 -- 172.21.15.40:0/4002094851 shutdown_connections 2023-10-16T08:39:44.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.767+0000 7f0ac52eb640 1 --2- 172.21.15.40:0/4002094851 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ac015af20 0x7f0ac0070a10 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:44.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.767+0000 7f0ac52eb640 1 --2- 172.21.15.40:0/4002094851 >> v2:172.21.15.86:3300/0 conn(0x7f0ac015ab40 0x7f0ac00cf3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:44.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.767+0000 7f0ac52eb640 1 -- 172.21.15.40:0/4002094851 >> 172.21.15.40:0/4002094851 conn(0x7f0ac00ce8b0 msgr2=0x7f0ac00cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:44.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.767+0000 7f0ac52eb640 1 -- 172.21.15.40:0/4002094851 shutdown_connections 2023-10-16T08:39:44.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.767+0000 7f0ac52eb640 1 -- 172.21.15.40:0/4002094851 wait complete. 2023-10-16T08:39:44.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.767+0000 7f0ac52eb640 1 Processor -- start 2023-10-16T08:39:44.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.767+0000 7f0ac52eb640 1 -- start start 2023-10-16T08:39:44.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.767+0000 7f0ac52eb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ac00b41e0 0x7f0ac0151e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:44.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ac52eb640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac015ab40 0x7f0ac0156e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:44.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ac52eb640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ac015af20 0x7f0ac00c6c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:44.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ac52eb640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00c5980 con 0x7f0ac015ab40 2023-10-16T08:39:44.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ac52eb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac00c67e0 con 0x7f0ac00b41e0 2023-10-16T08:39:44.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ac52eb640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ac0155980 con 0x7f0ac015af20 2023-10-16T08:39:44.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ab67fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac015ab40 0x7f0ac0156e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:44.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ab67fc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac015ab40 0x7f0ac0156e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35540/0 (socket says 172.21.15.40:35540) 2023-10-16T08:39:44.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ab67fc640 1 -- 172.21.15.40:0/2212842950 learned_addr learned my addr 172.21.15.40:0/2212842950 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:44.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0abeffd640 1 --2- 172.21.15.40:0/2212842950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ac00b41e0 0x7f0ac0151e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:44.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0abf7fe640 1 --2- 172.21.15.40:0/2212842950 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ac015af20 0x7f0ac00c6c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:44.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ab67fc640 1 -- 172.21.15.40:0/2212842950 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ac015af20 msgr2=0x7f0ac00c6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:44.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ab67fc640 1 --2- 172.21.15.40:0/2212842950 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0ac015af20 0x7f0ac00c6c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:44.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ab67fc640 1 -- 172.21.15.40:0/2212842950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ac00b41e0 msgr2=0x7f0ac0151e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:44.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ab67fc640 1 --2- 172.21.15.40:0/2212842950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0ac00b41e0 0x7f0ac0151e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:44.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ab67fc640 1 -- 172.21.15.40:0/2212842950 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa8009420 con 0x7f0ac015ab40 2023-10-16T08:39:44.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ab67fc640 1 --2- 172.21.15.40:0/2212842950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac015ab40 0x7f0ac0156e70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0aa801aa70 tx=0x7f0aa8019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:44.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ab7fff640 1 -- 172.21.15.40:0/2212842950 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa80ac620 con 0x7f0ac015ab40 2023-10-16T08:39:44.787 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ab7fff640 1 -- 172.21.15.40:0/2212842950 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa80ac8f0 con 0x7f0ac015ab40 2023-10-16T08:39:44.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ab7fff640 1 -- 172.21.15.40:0/2212842950 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa80acc60 con 0x7f0ac015ab40 2023-10-16T08:39:44.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ac52eb640 1 -- 172.21.15.40:0/2212842950 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ac00c5bd0 con 0x7f0ac015ab40 2023-10-16T08:39:44.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ac52eb640 1 -- 172.21.15.40:0/2212842950 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ac00c6190 con 0x7f0ac015ab40 2023-10-16T08:39:44.788 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ac52eb640 1 -- 172.21.15.40:0/2212842950 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f0ac0071360 con 0x7f0ac015ab40 2023-10-16T08:39:44.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ab7fff640 1 -- 172.21.15.40:0/2212842950 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f0aa80b3070 con 0x7f0ac015ab40 2023-10-16T08:39:44.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ab7fff640 1 --2- 172.21.15.40:0/2212842950 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0aa40664a0 0x7f0aa4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:44.789 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ab7fff640 1 -- 172.21.15.40:0/2212842950 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f0aa811e720 con 0x7f0ac015ab40 2023-10-16T08:39:44.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ab7fff640 1 --2- 172.21.15.40:0/2212842950 >> [v2:172.21.15.40:6816/3244406559,v1:172.21.15.40:6818/3244406559] conn(0x7f0aa406e820 0x7f0aa4070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:44.790 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ab7fff640 1 -- 172.21.15.40:0/2212842950 --> [v2:172.21.15.40:6816/3244406559,v1:172.21.15.40:6818/3244406559] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f0aa4071390 con 0x7f0aa406e820 2023-10-16T08:39:44.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0ab7fff640 1 -- 172.21.15.40:0/2212842950 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0aa811eb70 con 0x7f0ac015ab40 2023-10-16T08:39:44.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0abeffd640 1 --2- 172.21.15.40:0/2212842950 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0aa40664a0 0x7f0aa4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:44.791 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0abf7fe640 1 --2- 172.21.15.40:0/2212842950 >> [v2:172.21.15.40:6816/3244406559,v1:172.21.15.40:6818/3244406559] conn(0x7f0aa406e820 0x7f0aa4070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:44.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.771+0000 7f0abf7fe640 1 --2- 172.21.15.40:0/2212842950 >> [v2:172.21.15.40:6816/3244406559,v1:172.21.15.40:6818/3244406559] conn(0x7f0aa406e820 0x7f0aa4070cc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:44.792 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.775+0000 7f0abeffd640 1 --2- 172.21.15.40:0/2212842950 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0aa40664a0 0x7f0aa4068960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009aa0 tx=0x7f0ab0006f90 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:44.793 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.775+0000 7f0ab7fff640 1 -- 172.21.15.40:0/2212842950 <== osd.0 v2:172.21.15.40:6816/3244406559 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f0aa4071390 con 0x7f0aa406e820 2023-10-16T08:39:44.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.787+0000 7f0ac52eb640 1 -- 172.21.15.40:0/2212842950 --> [v2:172.21.15.40:6816/3244406559,v1:172.21.15.40:6818/3244406559] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f0ac015f130 con 0x7f0aa406e820 2023-10-16T08:39:44.794 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.787+0000 7f0ab7fff640 1 -- 172.21.15.40:0/2212842950 <== osd.0 v2:172.21.15.40:6816/3244406559 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f0ab80070f0 con 0x7f0aa406e820 2023-10-16T08:39:44.795 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-16T08:39:44.795 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T08:39:44.796 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-16T08:39:44.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.787+0000 7f0ac52eb640 1 -- 172.21.15.40:0/2212842950 >> [v2:172.21.15.40:6816/3244406559,v1:172.21.15.40:6818/3244406559] conn(0x7f0aa406e820 msgr2=0x7f0aa4070cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:44.797 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.787+0000 7f0ac52eb640 1 --2- 172.21.15.40:0/2212842950 >> [v2:172.21.15.40:6816/3244406559,v1:172.21.15.40:6818/3244406559] conn(0x7f0aa406e820 0x7f0aa4070cc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:44.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.787+0000 7f0ac52eb640 1 -- 172.21.15.40:0/2212842950 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0aa40664a0 msgr2=0x7f0aa4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:44.798 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.787+0000 7f0ac52eb640 1 --2- 172.21.15.40:0/2212842950 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f0aa40664a0 0x7f0aa4068960 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f0ab0009aa0 tx=0x7f0ab0006f90 comp rx=0 tx=0).stop 2023-10-16T08:39:44.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.787+0000 7f0ac52eb640 1 -- 172.21.15.40:0/2212842950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac015ab40 msgr2=0x7f0ac0156e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:44.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.787+0000 7f0ac52eb640 1 --2- 172.21.15.40:0/2212842950 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0ac015ab40 0x7f0ac0156e70 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f0aa801aa70 tx=0x7f0aa8019160 comp rx=0 tx=0).stop 2023-10-16T08:39:44.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.787+0000 7f0abf7fe640 1 -- 172.21.15.40:0/2212842950 reap_dead start 2023-10-16T08:39:44.799 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.787+0000 7f0ac52eb640 1 -- 172.21.15.40:0/2212842950 shutdown_connections 2023-10-16T08:39:44.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.787+0000 7f0ac52eb640 1 -- 172.21.15.40:0/2212842950 >> 172.21.15.40:0/2212842950 conn(0x7f0ac00ce8b0 msgr2=0x7f0ac00c8eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:44.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.787+0000 7f0ac52eb640 1 -- 172.21.15.40:0/2212842950 shutdown_connections 2023-10-16T08:39:44.800 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.787+0000 7f0ac52eb640 1 -- 172.21.15.40:0/2212842950 wait complete. 2023-10-16T08:39:44.809 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-16T08:39:44.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.919+0000 7fbca5341640 1 Processor -- start 2023-10-16T08:39:44.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.919+0000 7fbca5341640 1 -- start start 2023-10-16T08:39:44.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.919+0000 7fbca5341640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbca00af6d0 0x7fbca00cf310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:44.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.919+0000 7fbca5341640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbca00afc20 0x7fbca00cf850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:44.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.919+0000 7fbca5341640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca006fc20 con 0x7fbca015b2e0 2023-10-16T08:39:44.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.919+0000 7fbca5341640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca00cb380 con 0x7fbca00af6d0 2023-10-16T08:39:44.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.919+0000 7fbca5341640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca00b3200 con 0x7fbca00afc20 2023-10-16T08:39:44.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.919+0000 7fbc9e7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbca00af6d0 0x7fbca00cf310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:44.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.919+0000 7fbc9e7fc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbca00af6d0 0x7fbca00cf310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35552/0 (socket says 172.21.15.40:35552) 2023-10-16T08:39:44.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.919+0000 7fbc9effd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fbca015b2e0 0x7fbca00cc830 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34020/0 (socket says 172.21.15.40:34020) 2023-10-16T08:39:44.929 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.919+0000 7fbc9e7fc640 1 -- 172.21.15.40:0/1077385941 learned_addr learned my addr 172.21.15.40:0/1077385941 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:44.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.919+0000 7fbc9f7fe640 1 --2- 172.21.15.40:0/1077385941 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbca00afc20 0x7fbca00cf850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:44.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.923+0000 7fbc9e7fc640 1 -- 172.21.15.40:0/1077385941 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbca00afc20 msgr2=0x7fbca00cf850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:44.930 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.923+0000 7fbc9e7fc640 1 --2- 172.21.15.40:0/1077385941 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbca00afc20 0x7fbca00cf850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:44.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.923+0000 7fbc9e7fc640 1 -- 172.21.15.40:0/1077385941 >> v1:172.21.15.40:6789/0 conn(0x7fbca015b2e0 legacy=0x7fbca00cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:44.931 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.923+0000 7fbc9e7fc640 1 -- 172.21.15.40:0/1077385941 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca006fee0 con 0x7fbca00af6d0 2023-10-16T08:39:44.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.923+0000 7fbc9e7fc640 1 --2- 172.21.15.40:0/1077385941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca00af6d0 0x7fbca00cf310 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009870 tx=0x7fbc9401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9616765be111b017 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:44.935 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.923+0000 7fbc9dffb640 1 -- 172.21.15.40:0/1077385941 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc940aa610 con 0x7fbca00af6d0 2023-10-16T08:39:44.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.923+0000 7fbc9dffb640 1 -- 172.21.15.40:0/1077385941 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc940a6040 con 0x7fbca00af6d0 2023-10-16T08:39:44.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.923+0000 7fbc9dffb640 1 -- 172.21.15.40:0/1077385941 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc940aa990 con 0x7fbca00af6d0 2023-10-16T08:39:44.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.923+0000 7fbca5341640 1 -- 172.21.15.40:0/1077385941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca00af6d0 msgr2=0x7fbca00cf310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:44.936 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.923+0000 7fbca5341640 1 --2- 172.21.15.40:0/1077385941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca00af6d0 0x7fbca00cf310 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fbc94009870 tx=0x7fbc9401c4b0 comp rx=0 tx=0).stop 2023-10-16T08:39:44.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.923+0000 7fbca5341640 1 -- 172.21.15.40:0/1077385941 shutdown_connections 2023-10-16T08:39:44.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.923+0000 7fbca5341640 1 --2- 172.21.15.40:0/1077385941 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbca00afc20 0x7fbca00cf850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:44.937 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.923+0000 7fbca5341640 1 --2- 172.21.15.40:0/1077385941 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca00af6d0 0x7fbca00cf310 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:44.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.923+0000 7fbca5341640 1 -- 172.21.15.40:0/1077385941 >> 172.21.15.40:0/1077385941 conn(0x7fbca006d700 msgr2=0x7fbca006db10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:44.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.923+0000 7fbca5341640 1 -- 172.21.15.40:0/1077385941 shutdown_connections 2023-10-16T08:39:44.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.923+0000 7fbca5341640 1 -- 172.21.15.40:0/1077385941 wait complete. 2023-10-16T08:39:44.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbca5341640 1 Processor -- start 2023-10-16T08:39:44.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbca5341640 1 -- start start 2023-10-16T08:39:44.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbca5341640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca00af6d0 0x7fbca00b1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:44.939 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbca5341640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbca00afc20 0x7fbca0148c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbca5341640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbca0164a70 0x7fbca0164f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbca5341640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca00b12f0 con 0x7fbca00af6d0 2023-10-16T08:39:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbca5341640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca0165440 con 0x7fbca0164a70 2023-10-16T08:39:44.940 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbca5341640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca00b0460 con 0x7fbca00afc20 2023-10-16T08:39:44.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbc9effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca00af6d0 0x7fbca00b1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:44.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbc9effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca00af6d0 0x7fbca00b1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35556/0 (socket says 172.21.15.40:35556) 2023-10-16T08:39:44.941 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbc9effd640 1 -- 172.21.15.40:0/3315869652 learned_addr learned my addr 172.21.15.40:0/3315869652 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:44.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbc9e7fc640 1 --2- 172.21.15.40:0/3315869652 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbca00afc20 0x7fbca0148c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:44.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbc9f7fe640 1 --2- 172.21.15.40:0/3315869652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbca0164a70 0x7fbca0164f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:44.942 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbc9effd640 1 -- 172.21.15.40:0/3315869652 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbca00afc20 msgr2=0x7fbca0148c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:44.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbc9effd640 1 --2- 172.21.15.40:0/3315869652 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbca00afc20 0x7fbca0148c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:44.943 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbc9effd640 1 -- 172.21.15.40:0/3315869652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbca0164a70 msgr2=0x7fbca0164f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:44.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbc9effd640 1 --2- 172.21.15.40:0/3315869652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbca0164a70 0x7fbca0164f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:44.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbc9effd640 1 -- 172.21.15.40:0/3315869652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc940094d0 con 0x7fbca00af6d0 2023-10-16T08:39:44.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbc9f7fe640 1 --2- 172.21.15.40:0/3315869652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbca0164a70 0x7fbca0164f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:44.944 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbc9effd640 1 --2- 172.21.15.40:0/3315869652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca00af6d0 0x7fbca00b1d20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000c780 tx=0x7fbc9000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:44.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbc877fe640 1 -- 172.21.15.40:0/3315869652 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9000ce90 con 0x7fbca00af6d0 2023-10-16T08:39:44.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbc877fe640 1 -- 172.21.15.40:0/3315869652 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc9000a640 con 0x7fbca00af6d0 2023-10-16T08:39:44.945 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbc877fe640 1 -- 172.21.15.40:0/3315869652 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc9000a9b0 con 0x7fbca00af6d0 2023-10-16T08:39:44.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbca5341640 1 -- 172.21.15.40:0/3315869652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca00b2260 con 0x7fbca00af6d0 2023-10-16T08:39:44.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbca5341640 1 -- 172.21.15.40:0/3315869652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca00b0950 con 0x7fbca00af6d0 2023-10-16T08:39:44.946 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbca5341640 1 -- 172.21.15.40:0/3315869652 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca006f7b0 con 0x7fbca00af6d0 2023-10-16T08:39:44.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.927+0000 7fbc877fe640 1 -- 172.21.15.40:0/3315869652 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbc9000ab10 con 0x7fbca00af6d0 2023-10-16T08:39:44.947 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.931+0000 7fbc877fe640 1 --2- 172.21.15.40:0/3315869652 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbc8c0664a0 0x7fbc8c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:44.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.931+0000 7fbc877fe640 1 -- 172.21.15.40:0/3315869652 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fbc90083680 con 0x7fbca00af6d0 2023-10-16T08:39:44.948 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.931+0000 7fbc9e7fc640 1 --2- 172.21.15.40:0/3315869652 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbc8c0664a0 0x7fbc8c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:44.949 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.931+0000 7fbc9e7fc640 1 --2- 172.21.15.40:0/3315869652 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbc8c0664a0 0x7fbc8c068960 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbc940ae000 tx=0x7fbc94008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:44.950 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:44.935+0000 7fbc877fe640 1 -- 172.21.15.40:0/3315869652 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fbc900545b0 con 0x7fbca00af6d0 2023-10-16T08:39:45.096 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.091+0000 7fbca5341640 1 -- 172.21.15.40:0/3315869652 --> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fbca00ccf40 con 0x7fbc8c0664a0 2023-10-16T08:39:45.097 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.091+0000 7fbc877fe640 1 -- 172.21.15.40:0/3315869652 <== mgr.4111 v2:172.21.15.86:6832/1188970899 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbc940ba090 con 0x7fbc8c0664a0 2023-10-16T08:39:45.098 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 0 to deep-scrub 2023-10-16T08:39:45.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.095+0000 7fbca5341640 1 -- 172.21.15.40:0/3315869652 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbc8c0664a0 msgr2=0x7fbc8c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.095+0000 7fbca5341640 1 --2- 172.21.15.40:0/3315869652 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbc8c0664a0 0x7fbc8c068960 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fbc940ae000 tx=0x7fbc94008040 comp rx=0 tx=0).stop 2023-10-16T08:39:45.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.095+0000 7fbca5341640 1 -- 172.21.15.40:0/3315869652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca00af6d0 msgr2=0x7fbca00b1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.095+0000 7fbca5341640 1 --2- 172.21.15.40:0/3315869652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca00af6d0 0x7fbca00b1d20 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fbc9000c780 tx=0x7fbc9000cc40 comp rx=0 tx=0).stop 2023-10-16T08:39:45.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.095+0000 7fbca5341640 1 -- 172.21.15.40:0/3315869652 shutdown_connections 2023-10-16T08:39:45.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.095+0000 7fbca5341640 1 --2- 172.21.15.40:0/3315869652 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbca0164a70 0x7fbca0164f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.095+0000 7fbca5341640 1 --2- 172.21.15.40:0/3315869652 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbca00afc20 0x7fbca0148c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.095+0000 7fbca5341640 1 --2- 172.21.15.40:0/3315869652 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbc8c0664a0 0x7fbc8c068960 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.095+0000 7fbca5341640 1 --2- 172.21.15.40:0/3315869652 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbca00af6d0 0x7fbca00b1d20 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.095+0000 7fbca5341640 1 -- 172.21.15.40:0/3315869652 >> 172.21.15.40:0/3315869652 conn(0x7fbca006d700 msgr2=0x7fbca00cd9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:45.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.095+0000 7fbca5341640 1 -- 172.21.15.40:0/3315869652 shutdown_connections 2023-10-16T08:39:45.103 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.095+0000 7fbca5341640 1 -- 172.21.15.40:0/3315869652 wait complete. 2023-10-16T08:39:45.115 INFO:tasks.ceph:Scrubbing osd.1 2023-10-16T08:39:45.116 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-16T08:39:45.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.243+0000 7f40ffcd4640 1 Processor -- start 2023-10-16T08:39:45.248 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.243+0000 7f40ffcd4640 1 -- start start 2023-10-16T08:39:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.243+0000 7f40ffcd4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40f815e370 0x7f40f80ce7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.243+0000 7f40ffcd4640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f80d1510 con 0x7f40f80b1a40 2023-10-16T08:39:45.250 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.243+0000 7f40ffcd4640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40f80ccbf0 con 0x7f40f80cd400 2023-10-16T08:39:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.243+0000 7f40ffcd4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f80d1760 con 0x7f40f815e370 2023-10-16T08:39:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.243+0000 7f40fd26f640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f40f80b1a40 0x7f40f80cc4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34036/0 (socket says 172.21.15.40:34036) 2023-10-16T08:39:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.243+0000 7f40fd26f640 1 -- 172.21.15.40:0/338418780 learned_addr learned my addr 172.21.15.40:0/338418780 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:45.251 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.243+0000 7f40fda70640 1 --2- 172.21.15.40:0/338418780 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40f815e370 0x7f40f80ce7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.243+0000 7f40fca6e640 1 -- 172.21.15.40:0/338418780 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 281842761 0 0) 0x7f40f80d1510 con 0x7f40f80b1a40 2023-10-16T08:39:45.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.243+0000 7f40fca6e640 1 -- 172.21.15.40:0/338418780 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f40e4003530 con 0x7f40f80b1a40 2023-10-16T08:39:45.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.243+0000 7f40fca6e640 1 -- 172.21.15.40:0/338418780 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3893408980 0 0) 0x7f40f80ccbf0 con 0x7f40f80cd400 2023-10-16T08:39:45.252 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.243+0000 7f40fda70640 1 -- 172.21.15.40:0/338418780 >> v1:172.21.15.86:6789/0 conn(0x7f40f80cd400 legacy=0x7f40f80b3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.243+0000 7f40fda70640 1 -- 172.21.15.40:0/338418780 >> v1:172.21.15.40:6789/0 conn(0x7f40f80b1a40 legacy=0x7f40f80cc4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.253 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.243+0000 7f40fda70640 1 -- 172.21.15.40:0/338418780 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f80cb720 con 0x7f40f815e370 2023-10-16T08:39:45.254 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.247+0000 7f40fda70640 1 --2- 172.21.15.40:0/338418780 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40f815e370 0x7f40f80ce7d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f40f40097f0 tx=0x7f40f401c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f194d5befe6a5026 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:45.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.247+0000 7f40fca6e640 1 -- 172.21.15.40:0/338418780 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f40ab600 con 0x7f40f815e370 2023-10-16T08:39:45.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.247+0000 7f40fca6e640 1 -- 172.21.15.40:0/338418780 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f40a7050 con 0x7f40f815e370 2023-10-16T08:39:45.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.247+0000 7f40fca6e640 1 -- 172.21.15.40:0/338418780 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f40ab960 con 0x7f40f815e370 2023-10-16T08:39:45.255 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.247+0000 7f40ffcd4640 1 -- 172.21.15.40:0/338418780 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40f815e370 msgr2=0x7f40f80ce7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.247+0000 7f40ffcd4640 1 --2- 172.21.15.40:0/338418780 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40f815e370 0x7f40f80ce7d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f40f40097f0 tx=0x7f40f401c430 comp rx=0 tx=0).stop 2023-10-16T08:39:45.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.247+0000 7f40ffcd4640 1 -- 172.21.15.40:0/338418780 shutdown_connections 2023-10-16T08:39:45.256 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.247+0000 7f40ffcd4640 1 --2- 172.21.15.40:0/338418780 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40f815e370 0x7f40f80ce7d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.247+0000 7f40ffcd4640 1 -- 172.21.15.40:0/338418780 >> 172.21.15.40:0/338418780 conn(0x7f40f80d09f0 msgr2=0x7f40f80d0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:45.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.247+0000 7f40ffcd4640 1 -- 172.21.15.40:0/338418780 shutdown_connections 2023-10-16T08:39:45.257 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.247+0000 7f40ffcd4640 1 -- 172.21.15.40:0/338418780 wait complete. 2023-10-16T08:39:45.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.247+0000 7f40ffcd4640 1 Processor -- start 2023-10-16T08:39:45.258 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.247+0000 7f40ffcd4640 1 -- start start 2023-10-16T08:39:45.260 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40ffcd4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40f80b1a40 0x7f40f806adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40ffcd4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40f80cd400 0x7f40f815ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40ffcd4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f815e370 0x7f40f815b640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.261 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40ffcd4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f815c090 con 0x7f40f815e370 2023-10-16T08:39:45.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40ffcd4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40f806a960 con 0x7f40f80cd400 2023-10-16T08:39:45.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40ffcd4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40f806a030 con 0x7f40f80b1a40 2023-10-16T08:39:45.262 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40fe271640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f815e370 0x7f40f815b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40fe271640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f815e370 0x7f40f815b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35560/0 (socket says 172.21.15.40:35560) 2023-10-16T08:39:45.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40fe271640 1 -- 172.21.15.40:0/1118245108 learned_addr learned my addr 172.21.15.40:0/1118245108 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:45.263 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40fda70640 1 --2- 172.21.15.40:0/1118245108 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40f80b1a40 0x7f40f806adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40fd26f640 1 --2- 172.21.15.40:0/1118245108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40f80cd400 0x7f40f815ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40fe271640 1 -- 172.21.15.40:0/1118245108 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40f80b1a40 msgr2=0x7f40f806adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40fe271640 1 --2- 172.21.15.40:0/1118245108 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f40f80b1a40 0x7f40f806adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.264 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40fe271640 1 -- 172.21.15.40:0/1118245108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40f80cd400 msgr2=0x7f40f815ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40fe271640 1 --2- 172.21.15.40:0/1118245108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40f80cd400 0x7f40f815ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40fe271640 1 -- 172.21.15.40:0/1118245108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f4009420 con 0x7f40f815e370 2023-10-16T08:39:45.265 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40fe271640 1 --2- 172.21.15.40:0/1118245108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f815e370 0x7f40f815b640 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f40f000c780 tx=0x7f40f000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:45.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40ea7fc640 1 -- 172.21.15.40:0/1118245108 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f0003db0 con 0x7f40f815e370 2023-10-16T08:39:45.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40ffcd4640 1 -- 172.21.15.40:0/1118245108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40f80cdda0 con 0x7f40f815e370 2023-10-16T08:39:45.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40ea7fc640 1 -- 172.21.15.40:0/1118245108 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f000ce90 con 0x7f40f815e370 2023-10-16T08:39:45.266 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40ea7fc640 1 -- 172.21.15.40:0/1118245108 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f00052b0 con 0x7f40f815e370 2023-10-16T08:39:45.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40ffcd4640 1 -- 172.21.15.40:0/1118245108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40f806a5f0 con 0x7f40f815e370 2023-10-16T08:39:45.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40ffcd4640 1 -- 172.21.15.40:0/1118245108 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f40f80cdc20 con 0x7f40f815e370 2023-10-16T08:39:45.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40ea7fc640 1 -- 172.21.15.40:0/1118245108 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f40f0005410 con 0x7f40f815e370 2023-10-16T08:39:45.267 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40ea7fc640 1 --2- 172.21.15.40:0/1118245108 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f40dc0664a0 0x7f40dc068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40ea7fc640 1 -- 172.21.15.40:0/1118245108 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f40f0084800 con 0x7f40f815e370 2023-10-16T08:39:45.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40ea7fc640 1 --2- 172.21.15.40:0/1118245108 >> [v2:172.21.15.40:6817/2222522452,v1:172.21.15.40:6819/2222522452] conn(0x7f40dc06e820 0x7f40dc070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40ea7fc640 1 -- 172.21.15.40:0/1118245108 --> [v2:172.21.15.40:6817/2222522452,v1:172.21.15.40:6819/2222522452] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40dc071390 con 0x7f40dc06e820 2023-10-16T08:39:45.268 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40ea7fc640 1 -- 172.21.15.40:0/1118245108 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40f0084b70 con 0x7f40f815e370 2023-10-16T08:39:45.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40fd26f640 1 --2- 172.21.15.40:0/1118245108 >> [v2:172.21.15.40:6817/2222522452,v1:172.21.15.40:6819/2222522452] conn(0x7f40dc06e820 0x7f40dc070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40fda70640 1 --2- 172.21.15.40:0/1118245108 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f40dc0664a0 0x7f40dc068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.251+0000 7f40fd26f640 1 --2- 172.21.15.40:0/1118245108 >> [v2:172.21.15.40:6817/2222522452,v1:172.21.15.40:6819/2222522452] conn(0x7f40dc06e820 0x7f40dc070cc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:45.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.255+0000 7f40fda70640 1 --2- 172.21.15.40:0/1118245108 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f40dc0664a0 0x7f40dc068960 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f40f806cf30 tx=0x7f40f40a6000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:45.269 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.255+0000 7f40ea7fc640 1 -- 172.21.15.40:0/1118245108 <== osd.1 v2:172.21.15.40:6817/2222522452 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f40dc071390 con 0x7f40dc06e820 2023-10-16T08:39:45.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.267+0000 7f40ffcd4640 1 -- 172.21.15.40:0/1118245108 --> [v2:172.21.15.40:6817/2222522452,v1:172.21.15.40:6819/2222522452] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f40f80cbfc0 con 0x7f40dc06e820 2023-10-16T08:39:45.272 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.267+0000 7f40ea7fc640 1 -- 172.21.15.40:0/1118245108 <== osd.1 v2:172.21.15.40:6817/2222522452 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f40ec0017f0 con 0x7f40dc06e820 2023-10-16T08:39:45.273 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-16T08:39:45.273 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T08:39:45.274 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-16T08:39:45.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.267+0000 7f40ffcd4640 1 -- 172.21.15.40:0/1118245108 >> [v2:172.21.15.40:6817/2222522452,v1:172.21.15.40:6819/2222522452] conn(0x7f40dc06e820 msgr2=0x7f40dc070cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.267+0000 7f40ffcd4640 1 --2- 172.21.15.40:0/1118245108 >> [v2:172.21.15.40:6817/2222522452,v1:172.21.15.40:6819/2222522452] conn(0x7f40dc06e820 0x7f40dc070cc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.275 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.267+0000 7f40ffcd4640 1 -- 172.21.15.40:0/1118245108 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f40dc0664a0 msgr2=0x7f40dc068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.267+0000 7f40ffcd4640 1 --2- 172.21.15.40:0/1118245108 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f40dc0664a0 0x7f40dc068960 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f40f806cf30 tx=0x7f40f40a6000 comp rx=0 tx=0).stop 2023-10-16T08:39:45.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.267+0000 7f40ffcd4640 1 -- 172.21.15.40:0/1118245108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f815e370 msgr2=0x7f40f815b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.267+0000 7f40ffcd4640 1 --2- 172.21.15.40:0/1118245108 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f40f815e370 0x7f40f815b640 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f40f000c780 tx=0x7f40f000cc40 comp rx=0 tx=0).stop 2023-10-16T08:39:45.276 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.267+0000 7f40fe271640 1 -- 172.21.15.40:0/1118245108 reap_dead start 2023-10-16T08:39:45.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.267+0000 7f40ffcd4640 1 -- 172.21.15.40:0/1118245108 shutdown_connections 2023-10-16T08:39:45.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.267+0000 7f40ffcd4640 1 -- 172.21.15.40:0/1118245108 >> 172.21.15.40:0/1118245108 conn(0x7f40f80d09f0 msgr2=0x7f40f80cc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:45.277 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.267+0000 7f40ffcd4640 1 -- 172.21.15.40:0/1118245108 shutdown_connections 2023-10-16T08:39:45.278 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.267+0000 7f40ffcd4640 1 -- 172.21.15.40:0/1118245108 wait complete. 2023-10-16T08:39:45.289 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-16T08:39:45.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f674a607640 1 Processor -- start 2023-10-16T08:39:45.405 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f674a607640 1 -- start start 2023-10-16T08:39:45.407 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f674a607640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67440cae50 0x7f674406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f674a607640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f67440adec0 0x7f674406bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f674a607640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67440adca0 con 0x7f674415b040 2023-10-16T08:39:45.408 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f674a607640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67440c5840 con 0x7f67440adec0 2023-10-16T08:39:45.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f674a607640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67440cf890 con 0x7f67440cae50 2023-10-16T08:39:45.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6743fff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f674415b040 0x7f67440ca740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34046/0 (socket says 172.21.15.40:34046) 2023-10-16T08:39:45.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6743fff640 1 -- 172.21.15.40:0/118340004 learned_addr learned my addr 172.21.15.40:0/118340004 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:45.409 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f67437fe640 1 --2- 172.21.15.40:0/118340004 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67440cae50 0x7f674406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6748ba4640 1 --2- 172.21.15.40:0/118340004 >> v2:172.21.15.86:3300/0 conn(0x7f67440adec0 0x7f674406bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6742ffd640 1 -- 172.21.15.40:0/118340004 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2149695543 0 0) 0x7f67440adca0 con 0x7f674415b040 2023-10-16T08:39:45.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6742ffd640 1 -- 172.21.15.40:0/118340004 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6728003530 con 0x7f674415b040 2023-10-16T08:39:45.410 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6742ffd640 1 -- 172.21.15.40:0/118340004 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1061913251 0 0) 0x7f6728003530 con 0x7f674415b040 2023-10-16T08:39:45.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6742ffd640 1 -- 172.21.15.40:0/118340004 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67440cae50 msgr2=0x7f674406b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:45.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6742ffd640 1 --2- 172.21.15.40:0/118340004 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67440cae50 0x7f674406b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.411 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6742ffd640 1 -- 172.21.15.40:0/118340004 >> v2:172.21.15.86:3300/0 conn(0x7f67440adec0 msgr2=0x7f674406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:45.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6742ffd640 1 --2- 172.21.15.40:0/118340004 >> v2:172.21.15.86:3300/0 conn(0x7f67440adec0 0x7f674406bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6742ffd640 1 -- 172.21.15.40:0/118340004 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67440cf5b0 con 0x7f674415b040 2023-10-16T08:39:45.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6742ffd640 1 -- 172.21.15.40:0/118340004 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f6734001680 con 0x7f674415b040 2023-10-16T08:39:45.412 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6742ffd640 1 -- 172.21.15.40:0/118340004 >> v1:172.21.15.40:6789/0 conn(0x7f674415b040 legacy=0x7f67440ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6742ffd640 1 --2- 172.21.15.40:0/118340004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67280039f0 0x7f6728003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6742ffd640 1 --2- 172.21.15.40:0/118340004 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67440cae50 0x7f67280047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6742ffd640 1 --2- 172.21.15.40:0/118340004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6728004d10 0x7f6728005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.413 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6742ffd640 1 -- 172.21.15.40:0/118340004 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67440c5840 con 0x7f6728004d10 2023-10-16T08:39:45.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6742ffd640 1 -- 172.21.15.40:0/118340004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67440cf890 con 0x7f67280039f0 2023-10-16T08:39:45.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6742ffd640 1 -- 172.21.15.40:0/118340004 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f672800ba90 con 0x7f67440cae50 2023-10-16T08:39:45.414 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6748ba4640 1 --2- 172.21.15.40:0/118340004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6728004d10 0x7f6728005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f6743fff640 1 --2- 172.21.15.40:0/118340004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67280039f0 0x7f6728003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.399+0000 7f67437fe640 1 --2- 172.21.15.40:0/118340004 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67440cae50 0x7f67280047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.403+0000 7f6748ba4640 1 -- 172.21.15.40:0/118340004 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67440cae50 msgr2=0x7f67280047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:45.415 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.403+0000 7f6748ba4640 1 --2- 172.21.15.40:0/118340004 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f67440cae50 0x7f67280047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.403+0000 7f6748ba4640 1 -- 172.21.15.40:0/118340004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67280039f0 msgr2=0x7f6728003e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:45.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.403+0000 7f6748ba4640 1 --2- 172.21.15.40:0/118340004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67280039f0 0x7f6728003e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.416 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.403+0000 7f6748ba4640 1 -- 172.21.15.40:0/118340004 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6728003530 con 0x7f6728004d10 2023-10-16T08:39:45.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.403+0000 7f6748ba4640 1 --2- 172.21.15.40:0/118340004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6728004d10 0x7f6728005190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f673c00e170 tx=0x7f673c01cc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e14a7a833c3c2c18 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:45.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.403+0000 7f6742ffd640 1 -- 172.21.15.40:0/118340004 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c0a5b50 con 0x7f6728004d10 2023-10-16T08:39:45.420 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.403+0000 7f6742ffd640 1 -- 172.21.15.40:0/118340004 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673c00f040 con 0x7f6728004d10 2023-10-16T08:39:45.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.403+0000 7f6742ffd640 1 -- 172.21.15.40:0/118340004 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673c0ad440 con 0x7f6728004d10 2023-10-16T08:39:45.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.403+0000 7f674a607640 1 -- 172.21.15.40:0/118340004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6728004d10 msgr2=0x7f6728005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.403+0000 7f674a607640 1 --2- 172.21.15.40:0/118340004 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6728004d10 0x7f6728005190 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f673c00e170 tx=0x7f673c01cc50 comp rx=0 tx=0).stop 2023-10-16T08:39:45.421 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.403+0000 7f6748ba4640 1 -- 172.21.15.40:0/118340004 reap_dead start 2023-10-16T08:39:45.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.403+0000 7f674a607640 1 -- 172.21.15.40:0/118340004 shutdown_connections 2023-10-16T08:39:45.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.403+0000 7f674a607640 1 -- 172.21.15.40:0/118340004 >> 172.21.15.40:0/118340004 conn(0x7f67440ce960 msgr2=0x7f67440ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:45.422 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.403+0000 7f674a607640 1 -- 172.21.15.40:0/118340004 shutdown_connections 2023-10-16T08:39:45.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.403+0000 7f674a607640 1 -- 172.21.15.40:0/118340004 wait complete. 2023-10-16T08:39:45.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f674a607640 1 Processor -- start 2023-10-16T08:39:45.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f674a607640 1 -- start start 2023-10-16T08:39:45.423 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f674a607640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6728004d10 0x7f674414efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f674a607640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6744151ee0 0x7f674414e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f674a607640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6744150650 0x7f67441f69d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.424 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f674a607640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674414e7e0 con 0x7f6728004d10 2023-10-16T08:39:45.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f674a607640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6744154ef0 con 0x7f6744151ee0 2023-10-16T08:39:45.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f674a607640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6744152360 con 0x7f6744150650 2023-10-16T08:39:45.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f6743fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6728004d10 0x7f674414efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.425 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f6743fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6728004d10 0x7f674414efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35574/0 (socket says 172.21.15.40:35574) 2023-10-16T08:39:45.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f6743fff640 1 -- 172.21.15.40:0/2352302484 learned_addr learned my addr 172.21.15.40:0/2352302484 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:45.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f6748ba4640 1 --2- 172.21.15.40:0/2352302484 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6744150650 0x7f67441f69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.426 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f67437fe640 1 --2- 172.21.15.40:0/2352302484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6744151ee0 0x7f674414e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f6743fff640 1 -- 172.21.15.40:0/2352302484 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6744150650 msgr2=0x7f67441f69d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f6743fff640 1 --2- 172.21.15.40:0/2352302484 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6744150650 0x7f67441f69d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.427 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f6743fff640 1 -- 172.21.15.40:0/2352302484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6744151ee0 msgr2=0x7f674414e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f6743fff640 1 --2- 172.21.15.40:0/2352302484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6744151ee0 0x7f674414e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f6743fff640 1 -- 172.21.15.40:0/2352302484 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6744151c40 con 0x7f6728004d10 2023-10-16T08:39:45.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f6743fff640 1 --2- 172.21.15.40:0/2352302484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6728004d10 0x7f674414efc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f673400e780 tx=0x7f673400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:45.428 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f6740ff9640 1 -- 172.21.15.40:0/2352302484 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673400ee30 con 0x7f6728004d10 2023-10-16T08:39:45.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f6740ff9640 1 -- 172.21.15.40:0/2352302484 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f673400c8d0 con 0x7f6728004d10 2023-10-16T08:39:45.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f674a607640 1 -- 172.21.15.40:0/2352302484 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6744151680 con 0x7f6728004d10 2023-10-16T08:39:45.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f6740ff9640 1 -- 172.21.15.40:0/2352302484 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f673400cc80 con 0x7f6728004d10 2023-10-16T08:39:45.429 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f674a607640 1 -- 172.21.15.40:0/2352302484 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6744150130 con 0x7f6728004d10 2023-10-16T08:39:45.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f674a607640 1 -- 172.21.15.40:0/2352302484 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f674404e9c0 con 0x7f6728004d10 2023-10-16T08:39:45.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f6740ff9640 1 -- 172.21.15.40:0/2352302484 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f673401d460 con 0x7f6728004d10 2023-10-16T08:39:45.430 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.407+0000 7f6740ff9640 1 --2- 172.21.15.40:0/2352302484 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f672c0664a0 0x7f672c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.411+0000 7f6740ff9640 1 -- 172.21.15.40:0/2352302484 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6734083dc0 con 0x7f6728004d10 2023-10-16T08:39:45.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.411+0000 7f67437fe640 1 --2- 172.21.15.40:0/2352302484 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f672c0664a0 0x7f672c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.411+0000 7f67437fe640 1 --2- 172.21.15.40:0/2352302484 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f672c0664a0 0x7f672c068960 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f67441f7ad0 tx=0x7f6730008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:45.431 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.415+0000 7f6740ff9640 1 -- 172.21.15.40:0/2352302484 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f6734051240 con 0x7f6728004d10 2023-10-16T08:39:45.576 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.571+0000 7f674a607640 1 -- 172.21.15.40:0/2352302484 --> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f67440ad970 con 0x7f672c0664a0 2023-10-16T08:39:45.577 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.571+0000 7f6740ff9640 1 -- 172.21.15.40:0/2352302484 <== mgr.4111 v2:172.21.15.86:6832/1188970899 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f6730004030 con 0x7f672c0664a0 2023-10-16T08:39:45.577 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 1 to deep-scrub 2023-10-16T08:39:45.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.575+0000 7f674a607640 1 -- 172.21.15.40:0/2352302484 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f672c0664a0 msgr2=0x7f672c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.580 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.575+0000 7f674a607640 1 --2- 172.21.15.40:0/2352302484 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f672c0664a0 0x7f672c068960 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f67441f7ad0 tx=0x7f6730008040 comp rx=0 tx=0).stop 2023-10-16T08:39:45.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.575+0000 7f674a607640 1 -- 172.21.15.40:0/2352302484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6728004d10 msgr2=0x7f674414efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.575+0000 7f674a607640 1 --2- 172.21.15.40:0/2352302484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6728004d10 0x7f674414efc0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f673400e780 tx=0x7f673400ec40 comp rx=0 tx=0).stop 2023-10-16T08:39:45.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.575+0000 7f674a607640 1 -- 172.21.15.40:0/2352302484 shutdown_connections 2023-10-16T08:39:45.581 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.575+0000 7f674a607640 1 --2- 172.21.15.40:0/2352302484 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f6744150650 0x7f67441f69d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.575+0000 7f674a607640 1 --2- 172.21.15.40:0/2352302484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6744151ee0 0x7f674414e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.575+0000 7f674a607640 1 --2- 172.21.15.40:0/2352302484 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f672c0664a0 0x7f672c068960 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.582 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.575+0000 7f674a607640 1 --2- 172.21.15.40:0/2352302484 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f6728004d10 0x7f674414efc0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.575+0000 7f674a607640 1 -- 172.21.15.40:0/2352302484 >> 172.21.15.40:0/2352302484 conn(0x7f67440ce960 msgr2=0x7f67440b1580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:45.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.575+0000 7f674a607640 1 -- 172.21.15.40:0/2352302484 shutdown_connections 2023-10-16T08:39:45.583 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.575+0000 7f674a607640 1 -- 172.21.15.40:0/2352302484 wait complete. 2023-10-16T08:39:45.596 INFO:tasks.ceph:Scrubbing osd.2 2023-10-16T08:39:45.597 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-16T08:39:45.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.707+0000 7fdfec4d4640 1 Processor -- start 2023-10-16T08:39:45.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.707+0000 7fdfec4d4640 1 -- start start 2023-10-16T08:39:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfec4d4640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fdfe4069e90 0x7fdfe4071a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfec4d4640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdfe4071f80 0x7fdfe415c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfec4d4640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdfe40723d0 con 0x7fdfe40ccbd0 2023-10-16T08:39:45.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfec4d4640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe406bee0 con 0x7fdfe4071f80 2023-10-16T08:39:45.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfec4d4640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdfe406a370 con 0x7fdfe4069e90 2023-10-16T08:39:45.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfeaa71640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdfe4071f80 0x7fdfe415c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfeaa71640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fdfe4071f80 0x7fdfe415c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35590/0 (socket says 172.21.15.40:35590) 2023-10-16T08:39:45.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfeaa71640 1 -- 172.21.15.40:0/3744860361 learned_addr learned my addr 172.21.15.40:0/3744860361 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfe9a6f640 1 --2- 172.21.15.40:0/3744860361 >> v2:172.21.15.86:3300/0 conn(0x7fdfe4069e90 0x7fdfe4071a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfeaa71640 1 -- 172.21.15.40:0/3744860361 >> v2:172.21.15.86:3300/0 conn(0x7fdfe4069e90 msgr2=0x7fdfe4071a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfeaa71640 1 --2- 172.21.15.40:0/3744860361 >> v2:172.21.15.86:3300/0 conn(0x7fdfe4069e90 0x7fdfe4071a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfeaa71640 1 -- 172.21.15.40:0/3744860361 >> v1:172.21.15.86:6789/0 conn(0x7fdfe40ccbd0 legacy=0x7fdfe406b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.720 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfeaa71640 1 -- 172.21.15.40:0/3744860361 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfe406a650 con 0x7fdfe4071f80 2023-10-16T08:39:45.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfeaa71640 1 --2- 172.21.15.40:0/3744860361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfe4071f80 0x7fdfe415c5e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40caf50 tx=0x7fdfdc01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b76eec1bc8e55d71 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:45.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfe926e640 1 -- 172.21.15.40:0/3744860361 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc0090d0 con 0x7fdfe4071f80 2023-10-16T08:39:45.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfe926e640 1 -- 172.21.15.40:0/3744860361 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfdc009230 con 0x7fdfe4071f80 2023-10-16T08:39:45.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfe926e640 1 -- 172.21.15.40:0/3744860361 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc0a5a80 con 0x7fdfe4071f80 2023-10-16T08:39:45.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfec4d4640 1 -- 172.21.15.40:0/3744860361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfe4071f80 msgr2=0x7fdfe415c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfec4d4640 1 --2- 172.21.15.40:0/3744860361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfe4071f80 0x7fdfe415c5e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fdfe40caf50 tx=0x7fdfdc01c9a0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfec4d4640 1 -- 172.21.15.40:0/3744860361 shutdown_connections 2023-10-16T08:39:45.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfec4d4640 1 --2- 172.21.15.40:0/3744860361 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfe4071f80 0x7fdfe415c5e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfec4d4640 1 --2- 172.21.15.40:0/3744860361 >> v2:172.21.15.86:3300/0 conn(0x7fdfe4069e90 0x7fdfe4071a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfec4d4640 1 -- 172.21.15.40:0/3744860361 >> 172.21.15.40:0/3744860361 conn(0x7fdfe4065e80 msgr2=0x7fdfe4066290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:45.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfec4d4640 1 -- 172.21.15.40:0/3744860361 shutdown_connections 2023-10-16T08:39:45.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.711+0000 7fdfec4d4640 1 -- 172.21.15.40:0/3744860361 wait complete. 2023-10-16T08:39:45.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfec4d4640 1 Processor -- start 2023-10-16T08:39:45.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfec4d4640 1 -- start start 2023-10-16T08:39:45.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfec4d4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdfe4069e90 0x7fdfe415af40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfec4d4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfe4071f80 0x7fdfe4150340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfec4d4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfe40ccbd0 0x7fdfe415f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfec4d4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe40aec20 con 0x7fdfe40ccbd0 2023-10-16T08:39:45.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfec4d4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe415b480 con 0x7fdfe4071f80 2023-10-16T08:39:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfec4d4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdfe415b6b0 con 0x7fdfe4069e90 2023-10-16T08:39:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfeaa71640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfe40ccbd0 0x7fdfe415f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfeaa71640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfe40ccbd0 0x7fdfe415f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35606/0 (socket says 172.21.15.40:35606) 2023-10-16T08:39:45.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfeaa71640 1 -- 172.21.15.40:0/2780858338 learned_addr learned my addr 172.21.15.40:0/2780858338 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfea270640 1 --2- 172.21.15.40:0/2780858338 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdfe4069e90 0x7fdfe415af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfe9a6f640 1 --2- 172.21.15.40:0/2780858338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfe4071f80 0x7fdfe4150340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfeaa71640 1 -- 172.21.15.40:0/2780858338 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdfe4069e90 msgr2=0x7fdfe415af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfeaa71640 1 --2- 172.21.15.40:0/2780858338 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fdfe4069e90 0x7fdfe415af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfeaa71640 1 -- 172.21.15.40:0/2780858338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfe4071f80 msgr2=0x7fdfe4150340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfeaa71640 1 --2- 172.21.15.40:0/2780858338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfe4071f80 0x7fdfe4150340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfeaa71640 1 -- 172.21.15.40:0/2780858338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdfdc009b30 con 0x7fdfe40ccbd0 2023-10-16T08:39:45.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfe9a6f640 1 --2- 172.21.15.40:0/2780858338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdfe4071f80 0x7fdfe4150340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:45.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfeaa71640 1 --2- 172.21.15.40:0/2780858338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfe40ccbd0 0x7fdfe415f780 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc01a5d0 tx=0x7fdfdc019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:45.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfdaffd640 1 -- 172.21.15.40:0/2780858338 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc0b8880 con 0x7fdfe40ccbd0 2023-10-16T08:39:45.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfdaffd640 1 -- 172.21.15.40:0/2780858338 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdfdc0b89e0 con 0x7fdfe40ccbd0 2023-10-16T08:39:45.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfec4d4640 1 -- 172.21.15.40:0/2780858338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdfe415b990 con 0x7fdfe40ccbd0 2023-10-16T08:39:45.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfdaffd640 1 -- 172.21.15.40:0/2780858338 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc0b8d90 con 0x7fdfe40ccbd0 2023-10-16T08:39:45.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfec4d4640 1 -- 172.21.15.40:0/2780858338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdfe415bf50 con 0x7fdfe40ccbd0 2023-10-16T08:39:45.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.715+0000 7fdfec4d4640 1 -- 172.21.15.40:0/2780858338 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fdfe40ccfb0 con 0x7fdfe40ccbd0 2023-10-16T08:39:45.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.719+0000 7fdfdaffd640 1 -- 172.21.15.40:0/2780858338 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fdfdc0cd430 con 0x7fdfe40ccbd0 2023-10-16T08:39:45.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.719+0000 7fdfdaffd640 1 --2- 172.21.15.40:0/2780858338 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdfc80664a0 0x7fdfc8068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.735 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.719+0000 7fdfdaffd640 1 -- 172.21.15.40:0/2780858338 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fdfdc120040 con 0x7fdfe40ccbd0 2023-10-16T08:39:45.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.719+0000 7fdfdaffd640 1 --2- 172.21.15.40:0/2780858338 >> [v2:172.21.15.40:6800/2328483711,v1:172.21.15.40:6801/2328483711] conn(0x7fdfc806e820 0x7fdfc8070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.719+0000 7fdfdaffd640 1 -- 172.21.15.40:0/2780858338 --> [v2:172.21.15.40:6800/2328483711,v1:172.21.15.40:6801/2328483711] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fdfc8071390 con 0x7fdfc806e820 2023-10-16T08:39:45.736 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.719+0000 7fdfdaffd640 1 -- 172.21.15.40:0/2780858338 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdfdc120490 con 0x7fdfe40ccbd0 2023-10-16T08:39:45.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.719+0000 7fdfe9a6f640 1 --2- 172.21.15.40:0/2780858338 >> [v2:172.21.15.40:6800/2328483711,v1:172.21.15.40:6801/2328483711] conn(0x7fdfc806e820 0x7fdfc8070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.719+0000 7fdfea270640 1 --2- 172.21.15.40:0/2780858338 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdfc80664a0 0x7fdfc8068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.719+0000 7fdfe9a6f640 1 --2- 172.21.15.40:0/2780858338 >> [v2:172.21.15.40:6800/2328483711,v1:172.21.15.40:6801/2328483711] conn(0x7fdfc806e820 0x7fdfc8070cc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:45.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.719+0000 7fdfea270640 1 --2- 172.21.15.40:0/2780858338 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdfc80664a0 0x7fdfc8068960 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0000f70 tx=0x7fdfe0009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:45.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.719+0000 7fdfdaffd640 1 -- 172.21.15.40:0/2780858338 <== osd.2 v2:172.21.15.40:6800/2328483711 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fdfc8071390 con 0x7fdfc806e820 2023-10-16T08:39:45.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.731+0000 7fdfec4d4640 1 -- 172.21.15.40:0/2780858338 --> [v2:172.21.15.40:6800/2328483711,v1:172.21.15.40:6801/2328483711] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fdfe406aef0 con 0x7fdfc806e820 2023-10-16T08:39:45.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.731+0000 7fdfdaffd640 1 -- 172.21.15.40:0/2780858338 <== osd.2 v2:172.21.15.40:6800/2328483711 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fdfd4009520 con 0x7fdfc806e820 2023-10-16T08:39:45.739 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-16T08:39:45.740 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T08:39:45.740 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-16T08:39:45.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.731+0000 7fdfec4d4640 1 -- 172.21.15.40:0/2780858338 >> [v2:172.21.15.40:6800/2328483711,v1:172.21.15.40:6801/2328483711] conn(0x7fdfc806e820 msgr2=0x7fdfc8070cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.731+0000 7fdfec4d4640 1 --2- 172.21.15.40:0/2780858338 >> [v2:172.21.15.40:6800/2328483711,v1:172.21.15.40:6801/2328483711] conn(0x7fdfc806e820 0x7fdfc8070cc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.731+0000 7fdfec4d4640 1 -- 172.21.15.40:0/2780858338 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdfc80664a0 msgr2=0x7fdfc8068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.731+0000 7fdfec4d4640 1 --2- 172.21.15.40:0/2780858338 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fdfc80664a0 0x7fdfc8068960 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdfe0000f70 tx=0x7fdfe0009040 comp rx=0 tx=0).stop 2023-10-16T08:39:45.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.731+0000 7fdfec4d4640 1 -- 172.21.15.40:0/2780858338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfe40ccbd0 msgr2=0x7fdfe415f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.731+0000 7fdfec4d4640 1 --2- 172.21.15.40:0/2780858338 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fdfe40ccbd0 0x7fdfe415f780 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc01a5d0 tx=0x7fdfdc019740 comp rx=0 tx=0).stop 2023-10-16T08:39:45.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.731+0000 7fdfeaa71640 1 -- 172.21.15.40:0/2780858338 reap_dead start 2023-10-16T08:39:45.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.731+0000 7fdfec4d4640 1 -- 172.21.15.40:0/2780858338 shutdown_connections 2023-10-16T08:39:45.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.731+0000 7fdfec4d4640 1 -- 172.21.15.40:0/2780858338 >> 172.21.15.40:0/2780858338 conn(0x7fdfe4065e80 msgr2=0x7fdfe40cbc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:45.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.731+0000 7fdfec4d4640 1 -- 172.21.15.40:0/2780858338 shutdown_connections 2023-10-16T08:39:45.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.731+0000 7fdfec4d4640 1 -- 172.21.15.40:0/2780858338 wait complete. 2023-10-16T08:39:45.752 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-16T08:39:45.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f0096052640 1 Processor -- start 2023-10-16T08:39:45.860 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f0096052640 1 -- start start 2023-10-16T08:39:45.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f0096052640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f009015d180 0x7f009006b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.863 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f0096052640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f009015d560 0x7f009006fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f0096052640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0090070090 con 0x7f00900ad820 2023-10-16T08:39:45.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f0096052640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00900cf480 con 0x7f009015d180 2023-10-16T08:39:45.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f0096052640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00900cf760 con 0x7f009015d560 2023-10-16T08:39:45.864 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008ffff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f00900ad820 0x7f00900c9570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34050/0 (socket says 172.21.15.40:34050) 2023-10-16T08:39:45.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008ffff640 1 -- 172.21.15.40:0/3615685641 learned_addr learned my addr 172.21.15.40:0/3615685641 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:45.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008effd640 1 --2- 172.21.15.40:0/3615685641 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f009015d560 0x7f009006fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.865 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008f7fe640 1 --2- 172.21.15.40:0/3615685641 >> v2:172.21.15.86:3300/0 conn(0x7f009015d180 0x7f009006b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008e7fc640 1 -- 172.21.15.40:0/3615685641 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2742688941 0 0) 0x7f0090070090 con 0x7f00900ad820 2023-10-16T08:39:45.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008e7fc640 1 -- 172.21.15.40:0/3615685641 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0078003530 con 0x7f00900ad820 2023-10-16T08:39:45.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008e7fc640 1 -- 172.21.15.40:0/3615685641 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3702062240 0 0) 0x7f0078003530 con 0x7f00900ad820 2023-10-16T08:39:45.866 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008e7fc640 1 -- 172.21.15.40:0/3615685641 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f009015d560 msgr2=0x7f009006fb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:45.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008e7fc640 1 --2- 172.21.15.40:0/3615685641 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f009015d560 0x7f009006fb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008e7fc640 1 -- 172.21.15.40:0/3615685641 >> v2:172.21.15.86:3300/0 conn(0x7f009015d180 msgr2=0x7f009006b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:45.867 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008e7fc640 1 --2- 172.21.15.40:0/3615685641 >> v2:172.21.15.86:3300/0 conn(0x7f009015d180 0x7f009006b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008e7fc640 1 -- 172.21.15.40:0/3615685641 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00900c8fb0 con 0x7f00900ad820 2023-10-16T08:39:45.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008e7fc640 1 -- 172.21.15.40:0/3615685641 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7f0088001a00 con 0x7f00900ad820 2023-10-16T08:39:45.868 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008e7fc640 1 -- 172.21.15.40:0/3615685641 >> v1:172.21.15.40:6789/0 conn(0x7f00900ad820 legacy=0x7f00900c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008e7fc640 1 --2- 172.21.15.40:0/3615685641 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f009015d560 0x7f00780039f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008e7fc640 1 --2- 172.21.15.40:0/3615685641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0078003f70 0x7f00780047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008e7fc640 1 --2- 172.21.15.40:0/3615685641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0078004d10 0x7f0078005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.869 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008e7fc640 1 -- 172.21.15.40:0/3615685641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00900cf480 con 0x7f0078004d10 2023-10-16T08:39:45.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008e7fc640 1 -- 172.21.15.40:0/3615685641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00900cf760 con 0x7f0078003f70 2023-10-16T08:39:45.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008e7fc640 1 -- 172.21.15.40:0/3615685641 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007800b9c0 con 0x7f009015d560 2023-10-16T08:39:45.870 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008ffff640 1 --2- 172.21.15.40:0/3615685641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0078004d10 0x7f0078005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008effd640 1 --2- 172.21.15.40:0/3615685641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0078003f70 0x7f00780047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008f7fe640 1 --2- 172.21.15.40:0/3615685641 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f009015d560 0x7f00780039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008ffff640 1 -- 172.21.15.40:0/3615685641 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f009015d560 msgr2=0x7f00780039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:45.871 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008ffff640 1 --2- 172.21.15.40:0/3615685641 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f009015d560 0x7f00780039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008ffff640 1 -- 172.21.15.40:0/3615685641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0078003f70 msgr2=0x7f00780047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:45.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008ffff640 1 --2- 172.21.15.40:0/3615685641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0078003f70 0x7f00780047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.872 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008ffff640 1 -- 172.21.15.40:0/3615685641 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0078003530 con 0x7f0078004d10 2023-10-16T08:39:45.873 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.855+0000 7f008effd640 1 --2- 172.21.15.40:0/3615685641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0078003f70 0x7f00780047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:45.876 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.859+0000 7f008ffff640 1 --2- 172.21.15.40:0/3615685641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0078004d10 0x7f0078005190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f008800a8d0 tx=0x7f008801d3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8f0dda5b70ecee1 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:45.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.859+0000 7f008e7fc640 1 -- 172.21.15.40:0/3615685641 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00880ad630 con 0x7f0078004d10 2023-10-16T08:39:45.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.859+0000 7f008e7fc640 1 -- 172.21.15.40:0/3615685641 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00880a8070 con 0x7f0078004d10 2023-10-16T08:39:45.877 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.859+0000 7f008e7fc640 1 -- 172.21.15.40:0/3615685641 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00880ad9d0 con 0x7f0078004d10 2023-10-16T08:39:45.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.859+0000 7f0096052640 1 -- 172.21.15.40:0/3615685641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0078004d10 msgr2=0x7f0078005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.859+0000 7f0096052640 1 --2- 172.21.15.40:0/3615685641 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0078004d10 0x7f0078005190 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f008800a8d0 tx=0x7f008801d3e0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.878 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.859+0000 7f008ffff640 1 -- 172.21.15.40:0/3615685641 reap_dead start 2023-10-16T08:39:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.859+0000 7f0096052640 1 -- 172.21.15.40:0/3615685641 shutdown_connections 2023-10-16T08:39:45.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.859+0000 7f0096052640 1 -- 172.21.15.40:0/3615685641 >> 172.21.15.40:0/3615685641 conn(0x7f00900ce960 msgr2=0x7f00900ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.859+0000 7f0096052640 1 -- 172.21.15.40:0/3615685641 shutdown_connections 2023-10-16T08:39:45.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.859+0000 7f0096052640 1 -- 172.21.15.40:0/3615685641 wait complete. 2023-10-16T08:39:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.859+0000 7f0096052640 1 Processor -- start 2023-10-16T08:39:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.859+0000 7f0096052640 1 -- start start 2023-10-16T08:39:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f0096052640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0078004d10 0x7f009014f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.881 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f0096052640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00900c94d0 0x7f00900c9950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f0096052640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00901e46d0 0x7f00901e4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f0096052640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00901f42b0 con 0x7f00900c94d0 2023-10-16T08:39:45.882 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f0096052640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00900c88f0 con 0x7f00901e46d0 2023-10-16T08:39:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f0096052640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00900c9e90 con 0x7f0078004d10 2023-10-16T08:39:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f008effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00900c94d0 0x7f00900c9950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f008effd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00900c94d0 0x7f00900c9950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35626/0 (socket says 172.21.15.40:35626) 2023-10-16T08:39:45.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f008effd640 1 -- 172.21.15.40:0/3105215489 learned_addr learned my addr 172.21.15.40:0/3105215489 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:45.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f008f7fe640 1 --2- 172.21.15.40:0/3105215489 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0078004d10 0x7f009014f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f008effd640 1 -- 172.21.15.40:0/3105215489 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0078004d10 msgr2=0x7f009014f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f008effd640 1 --2- 172.21.15.40:0/3105215489 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0078004d10 0x7f009014f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f008effd640 1 -- 172.21.15.40:0/3105215489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00901e46d0 msgr2=0x7f00901e4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:45.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f008effd640 1 --2- 172.21.15.40:0/3105215489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00901e46d0 0x7f00901e4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:45.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f008effd640 1 -- 172.21.15.40:0/3105215489 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00901f39a0 con 0x7f00900c94d0 2023-10-16T08:39:45.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f008effd640 1 --2- 172.21.15.40:0/3105215489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00900c94d0 0x7f00900c9950 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f00840115f0 tx=0x7f0084011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:45.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f0077fff640 1 -- 172.21.15.40:0/3105215489 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084004220 con 0x7f00900c94d0 2023-10-16T08:39:45.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f0077fff640 1 -- 172.21.15.40:0/3105215489 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0084004380 con 0x7f00900c94d0 2023-10-16T08:39:45.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f0077fff640 1 -- 172.21.15.40:0/3105215489 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0084015870 con 0x7f00900c94d0 2023-10-16T08:39:45.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f0096052640 1 -- 172.21.15.40:0/3105215489 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00901e53a0 con 0x7f00900c94d0 2023-10-16T08:39:45.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f0096052640 1 -- 172.21.15.40:0/3105215489 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00901f3ed0 con 0x7f00900c94d0 2023-10-16T08:39:45.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f0096052640 1 -- 172.21.15.40:0/3105215489 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f009004e9c0 con 0x7f00900c94d0 2023-10-16T08:39:45.887 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f0077fff640 1 -- 172.21.15.40:0/3105215489 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f00840159d0 con 0x7f00900c94d0 2023-10-16T08:39:45.888 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f0077fff640 1 --2- 172.21.15.40:0/3105215489 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f007c0664d0 0x7f007c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:45.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f0077fff640 1 -- 172.21.15.40:0/3105215489 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f0084083fa0 con 0x7f00900c94d0 2023-10-16T08:39:45.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.863+0000 7f008f7fe640 1 --2- 172.21.15.40:0/3105215489 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f007c0664d0 0x7f007c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:45.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.867+0000 7f008f7fe640 1 --2- 172.21.15.40:0/3105215489 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f007c0664d0 0x7f007c068990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f00800099f0 tx=0x7f0080008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:45.889 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:45.867+0000 7f0077fff640 1 -- 172.21.15.40:0/3105215489 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f00840b53d0 con 0x7f00900c94d0 2023-10-16T08:39:46.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.022+0000 7f0096052640 1 -- 172.21.15.40:0/3105215489 --> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f00900ade70 con 0x7f007c0664d0 2023-10-16T08:39:46.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.026+0000 7f0077fff640 1 -- 172.21.15.40:0/3105215489 <== mgr.4111 v2:172.21.15.86:6832/1188970899 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0080004020 con 0x7f007c0664d0 2023-10-16T08:39:46.032 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 2 to deep-scrub 2023-10-16T08:39:46.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.026+0000 7f0096052640 1 -- 172.21.15.40:0/3105215489 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f007c0664d0 msgr2=0x7f007c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.026+0000 7f0096052640 1 --2- 172.21.15.40:0/3105215489 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f007c0664d0 0x7f007c068990 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f00800099f0 tx=0x7f0080008040 comp rx=0 tx=0).stop 2023-10-16T08:39:46.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.026+0000 7f0096052640 1 -- 172.21.15.40:0/3105215489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00900c94d0 msgr2=0x7f00900c9950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.026+0000 7f0096052640 1 --2- 172.21.15.40:0/3105215489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00900c94d0 0x7f00900c9950 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f00840115f0 tx=0x7f0084011ab0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.026+0000 7f0096052640 1 -- 172.21.15.40:0/3105215489 shutdown_connections 2023-10-16T08:39:46.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.026+0000 7f0096052640 1 --2- 172.21.15.40:0/3105215489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f00901e46d0 0x7f00901e4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.026+0000 7f0096052640 1 --2- 172.21.15.40:0/3105215489 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f007c0664d0 0x7f007c068990 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.026+0000 7f0096052640 1 --2- 172.21.15.40:0/3105215489 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f00900c94d0 0x7f00900c9950 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.026+0000 7f0096052640 1 --2- 172.21.15.40:0/3105215489 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f0078004d10 0x7f009014f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.026+0000 7f0096052640 1 -- 172.21.15.40:0/3105215489 >> 172.21.15.40:0/3105215489 conn(0x7f00900ce960 msgr2=0x7f00900cbc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:46.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.026+0000 7f0096052640 1 -- 172.21.15.40:0/3105215489 shutdown_connections 2023-10-16T08:39:46.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.026+0000 7f0096052640 1 -- 172.21.15.40:0/3105215489 wait complete. 2023-10-16T08:39:46.051 INFO:tasks.ceph:Scrubbing osd.3 2023-10-16T08:39:46.052 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-16T08:39:46.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb746950640 1 Processor -- start 2023-10-16T08:39:46.176 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb746950640 1 -- start start 2023-10-16T08:39:46.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb746950640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7400ce540 0x7fb7400cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb746950640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7400cd2e0 con 0x7fb7400d02c0 2023-10-16T08:39:46.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb746950640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7400cf930 con 0x7fb7400afb90 2023-10-16T08:39:46.180 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb746950640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400cc6d0 con 0x7fb7400ce540 2023-10-16T08:39:46.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb744eed640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb7400d02c0 0x7fb740164750 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34062/0 (socket says 172.21.15.40:34062) 2023-10-16T08:39:46.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb744eed640 1 -- 172.21.15.40:0/1822123901 learned_addr learned my addr 172.21.15.40:0/1822123901 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:46.181 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73ffff640 1 --2- 172.21.15.40:0/1822123901 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7400ce540 0x7fb7400cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73effd640 1 -- 172.21.15.40:0/1822123901 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 166430441 0 0) 0x7fb7400cd2e0 con 0x7fb7400d02c0 2023-10-16T08:39:46.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73effd640 1 -- 172.21.15.40:0/1822123901 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb728003530 con 0x7fb7400d02c0 2023-10-16T08:39:46.182 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73effd640 1 -- 172.21.15.40:0/1822123901 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2217288918 0 0) 0x7fb7400cf930 con 0x7fb7400afb90 2023-10-16T08:39:46.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73effd640 1 -- 172.21.15.40:0/1822123901 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb7400cd2e0 con 0x7fb7400afb90 2023-10-16T08:39:46.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73effd640 1 -- 172.21.15.40:0/1822123901 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 913793068 0 0) 0x7fb728003530 con 0x7fb7400d02c0 2023-10-16T08:39:46.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73effd640 1 -- 172.21.15.40:0/1822123901 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7400ce540 msgr2=0x7fb7400cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:46.183 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73effd640 1 --2- 172.21.15.40:0/1822123901 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7400ce540 0x7fb7400cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73effd640 1 -- 172.21.15.40:0/1822123901 >> v1:172.21.15.86:6789/0 conn(0x7fb7400afb90 legacy=0x7fb7400cfbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73effd640 1 -- 172.21.15.40:0/1822123901 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7400cc9b0 con 0x7fb7400d02c0 2023-10-16T08:39:46.184 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73effd640 1 -- 172.21.15.40:0/1822123901 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7fb738001a00 con 0x7fb7400d02c0 2023-10-16T08:39:46.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73effd640 1 -- 172.21.15.40:0/1822123901 >> v1:172.21.15.40:6789/0 conn(0x7fb7400d02c0 legacy=0x7fb740164750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73effd640 1 --2- 172.21.15.40:0/1822123901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb728003bb0 0x7fb728003fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73effd640 1 --2- 172.21.15.40:0/1822123901 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7400ce540 0x7fb7280047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.185 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73effd640 1 --2- 172.21.15.40:0/1822123901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb728004d10 0x7fb728005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73effd640 1 -- 172.21.15.40:0/1822123901 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400cc6d0 con 0x7fb728003bb0 2023-10-16T08:39:46.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73effd640 1 -- 172.21.15.40:0/1822123901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb72800bac0 con 0x7fb728004d10 2023-10-16T08:39:46.186 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73effd640 1 -- 172.21.15.40:0/1822123901 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb72800bc80 con 0x7fb7400ce540 2023-10-16T08:39:46.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73ffff640 1 --2- 172.21.15.40:0/1822123901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb728003bb0 0x7fb728003fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73f7fe640 1 --2- 172.21.15.40:0/1822123901 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7400ce540 0x7fb7280047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb744eed640 1 --2- 172.21.15.40:0/1822123901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb728004d10 0x7fb728005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.187 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73ffff640 1 -- 172.21.15.40:0/1822123901 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7400ce540 msgr2=0x7fb7280047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:46.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73ffff640 1 --2- 172.21.15.40:0/1822123901 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb7400ce540 0x7fb7280047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73ffff640 1 -- 172.21.15.40:0/1822123901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb728004d10 msgr2=0x7fb728005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:46.188 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73ffff640 1 --2- 172.21.15.40:0/1822123901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb728004d10 0x7fb728005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb73ffff640 1 -- 172.21.15.40:0/1822123901 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb728003530 con 0x7fb728003bb0 2023-10-16T08:39:46.189 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.170+0000 7fb744eed640 1 --2- 172.21.15.40:0/1822123901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb728004d10 0x7fb728005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.174+0000 7fb73ffff640 1 --2- 172.21.15.40:0/1822123901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb728003bb0 0x7fb728003fb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb730009ae0 tx=0x7fb73001c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35f0aeadc11dc0d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.174+0000 7fb73effd640 1 -- 172.21.15.40:0/1822123901 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7300a6420 con 0x7fb728003bb0 2023-10-16T08:39:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.174+0000 7fb73effd640 1 -- 172.21.15.40:0/1822123901 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7300a6580 con 0x7fb728003bb0 2023-10-16T08:39:46.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.174+0000 7fb73effd640 1 -- 172.21.15.40:0/1822123901 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7300a6930 con 0x7fb728003bb0 2023-10-16T08:39:46.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.174+0000 7fb746950640 1 -- 172.21.15.40:0/1822123901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb728003bb0 msgr2=0x7fb728003fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.174+0000 7fb746950640 1 --2- 172.21.15.40:0/1822123901 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb728003bb0 0x7fb728003fb0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb730009ae0 tx=0x7fb73001c930 comp rx=0 tx=0).stop 2023-10-16T08:39:46.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.174+0000 7fb744eed640 1 -- 172.21.15.40:0/1822123901 reap_dead start 2023-10-16T08:39:46.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.174+0000 7fb746950640 1 -- 172.21.15.40:0/1822123901 shutdown_connections 2023-10-16T08:39:46.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.174+0000 7fb746950640 1 -- 172.21.15.40:0/1822123901 >> 172.21.15.40:0/1822123901 conn(0x7fb740065e80 msgr2=0x7fb740066290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:46.195 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.174+0000 7fb746950640 1 -- 172.21.15.40:0/1822123901 shutdown_connections 2023-10-16T08:39:46.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.174+0000 7fb746950640 1 -- 172.21.15.40:0/1822123901 wait complete. 2023-10-16T08:39:46.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.174+0000 7fb746950640 1 Processor -- start 2023-10-16T08:39:46.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.174+0000 7fb746950640 1 -- start start 2023-10-16T08:39:46.196 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb746950640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb728003bb0 0x7fb74014ee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb746950640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb74015b1e0 0x7fb7400c9750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb746950640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb74015c7c0 0x7fb74015cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb746950640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74015d180 con 0x7fb728003bb0 2023-10-16T08:39:46.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb746950640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb74014f3b0 con 0x7fb74015b1e0 2023-10-16T08:39:46.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb746950640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7400ca840 con 0x7fb74015c7c0 2023-10-16T08:39:46.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb73ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb728003bb0 0x7fb74014ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb73ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb728003bb0 0x7fb74014ee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35646/0 (socket says 172.21.15.40:35646) 2023-10-16T08:39:46.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb73ffff640 1 -- 172.21.15.40:0/210486288 learned_addr learned my addr 172.21.15.40:0/210486288 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:46.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb744eed640 1 --2- 172.21.15.40:0/210486288 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb74015c7c0 0x7fb74015cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb73f7fe640 1 --2- 172.21.15.40:0/210486288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb74015b1e0 0x7fb7400c9750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb73ffff640 1 -- 172.21.15.40:0/210486288 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb74015c7c0 msgr2=0x7fb74015cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb73ffff640 1 --2- 172.21.15.40:0/210486288 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb74015c7c0 0x7fb74015cc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb73ffff640 1 -- 172.21.15.40:0/210486288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb74015b1e0 msgr2=0x7fb7400c9750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb73ffff640 1 --2- 172.21.15.40:0/210486288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb74015b1e0 0x7fb7400c9750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb73ffff640 1 -- 172.21.15.40:0/210486288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7400c9fa0 con 0x7fb728003bb0 2023-10-16T08:39:46.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb73ffff640 1 --2- 172.21.15.40:0/210486288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb728003bb0 0x7fb74014ee70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb730009ae0 tx=0x7fb728005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:46.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb73cff9640 1 -- 172.21.15.40:0/210486288 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7300a6da0 con 0x7fb728003bb0 2023-10-16T08:39:46.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb73cff9640 1 -- 172.21.15.40:0/210486288 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb730004020 con 0x7fb728003bb0 2023-10-16T08:39:46.201 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb746950640 1 -- 172.21.15.40:0/210486288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7400ca280 con 0x7fb728003bb0 2023-10-16T08:39:46.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb73cff9640 1 -- 172.21.15.40:0/210486288 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7300043d0 con 0x7fb728003bb0 2023-10-16T08:39:46.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb746950640 1 -- 172.21.15.40:0/210486288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7400c7ae0 con 0x7fb728003bb0 2023-10-16T08:39:46.202 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb746950640 1 -- 172.21.15.40:0/210486288 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fb7400cc3a0 con 0x7fb728003bb0 2023-10-16T08:39:46.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb73cff9640 1 -- 172.21.15.40:0/210486288 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb730003620 con 0x7fb728003bb0 2023-10-16T08:39:46.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb73cff9640 1 --2- 172.21.15.40:0/210486288 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb72c0664d0 0x7fb72c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.203 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb73cff9640 1 -- 172.21.15.40:0/210486288 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb73011dea0 con 0x7fb728003bb0 2023-10-16T08:39:46.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb73cff9640 1 --2- 172.21.15.40:0/210486288 >> [v2:172.21.15.40:6808/1155811160,v1:172.21.15.40:6809/1155811160] conn(0x7fb72c06e850 0x7fb72c070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb73cff9640 1 -- 172.21.15.40:0/210486288 --> [v2:172.21.15.40:6808/1155811160,v1:172.21.15.40:6809/1155811160] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fb72c0713c0 con 0x7fb72c06e850 2023-10-16T08:39:46.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb73cff9640 1 -- 172.21.15.40:0/210486288 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb73011e2f0 con 0x7fb728003bb0 2023-10-16T08:39:46.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb73f7fe640 1 --2- 172.21.15.40:0/210486288 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb72c0664d0 0x7fb72c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.178+0000 7fb744eed640 1 --2- 172.21.15.40:0/210486288 >> [v2:172.21.15.40:6808/1155811160,v1:172.21.15.40:6809/1155811160] conn(0x7fb72c06e850 0x7fb72c070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.182+0000 7fb744eed640 1 --2- 172.21.15.40:0/210486288 >> [v2:172.21.15.40:6808/1155811160,v1:172.21.15.40:6809/1155811160] conn(0x7fb72c06e850 0x7fb72c070cf0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:46.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.182+0000 7fb73f7fe640 1 --2- 172.21.15.40:0/210486288 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb72c0664d0 0x7fb72c068990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb74006bab0 tx=0x7fb734009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:46.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.182+0000 7fb73cff9640 1 -- 172.21.15.40:0/210486288 <== osd.3 v2:172.21.15.40:6808/1155811160 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fb72c0713c0 con 0x7fb72c06e850 2023-10-16T08:39:46.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.194+0000 7fb746950640 1 -- 172.21.15.40:0/210486288 --> [v2:172.21.15.40:6808/1155811160,v1:172.21.15.40:6809/1155811160] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fb74015d180 con 0x7fb72c06e850 2023-10-16T08:39:46.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.194+0000 7fb73cff9640 1 -- 172.21.15.40:0/210486288 <== osd.3 v2:172.21.15.40:6808/1155811160 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fb74015d180 con 0x7fb72c06e850 2023-10-16T08:39:46.208 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-16T08:39:46.208 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T08:39:46.208 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-16T08:39:46.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.194+0000 7fb746950640 1 -- 172.21.15.40:0/210486288 >> [v2:172.21.15.40:6808/1155811160,v1:172.21.15.40:6809/1155811160] conn(0x7fb72c06e850 msgr2=0x7fb72c070cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.194+0000 7fb746950640 1 --2- 172.21.15.40:0/210486288 >> [v2:172.21.15.40:6808/1155811160,v1:172.21.15.40:6809/1155811160] conn(0x7fb72c06e850 0x7fb72c070cf0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.194+0000 7fb746950640 1 -- 172.21.15.40:0/210486288 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb72c0664d0 msgr2=0x7fb72c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.194+0000 7fb746950640 1 --2- 172.21.15.40:0/210486288 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb72c0664d0 0x7fb72c068990 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fb74006bab0 tx=0x7fb734009040 comp rx=0 tx=0).stop 2023-10-16T08:39:46.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.194+0000 7fb746950640 1 -- 172.21.15.40:0/210486288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb728003bb0 msgr2=0x7fb74014ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.194+0000 7fb746950640 1 --2- 172.21.15.40:0/210486288 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb728003bb0 0x7fb74014ee70 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb730009ae0 tx=0x7fb728005800 comp rx=0 tx=0).stop 2023-10-16T08:39:46.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.194+0000 7fb744eed640 1 -- 172.21.15.40:0/210486288 reap_dead start 2023-10-16T08:39:46.211 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.194+0000 7fb746950640 1 -- 172.21.15.40:0/210486288 shutdown_connections 2023-10-16T08:39:46.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.194+0000 7fb746950640 1 -- 172.21.15.40:0/210486288 >> 172.21.15.40:0/210486288 conn(0x7fb740065e80 msgr2=0x7fb740159b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:46.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.194+0000 7fb746950640 1 -- 172.21.15.40:0/210486288 shutdown_connections 2023-10-16T08:39:46.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.194+0000 7fb746950640 1 -- 172.21.15.40:0/210486288 wait complete. 2023-10-16T08:39:46.216 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-16T08:39:46.333 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.326+0000 7fea29f62640 1 Processor -- start 2023-10-16T08:39:46.334 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.326+0000 7fea29f62640 1 -- start start 2023-10-16T08:39:46.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.326+0000 7fea29f62640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea240cca60 0x7fea240b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.335 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.326+0000 7fea29f62640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fea240cced0 0x7fea240b2d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.326+0000 7fea29f62640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea24071790 con 0x7fea2415ad30 2023-10-16T08:39:46.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.326+0000 7fea29f62640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea2406e480 con 0x7fea240cced0 2023-10-16T08:39:46.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.326+0000 7fea29f62640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea240cc350 con 0x7fea240cca60 2023-10-16T08:39:46.336 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.326+0000 7fea23fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fea240cced0 0x7fea240b2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.326+0000 7fea23fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fea240cced0 0x7fea240b2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35656/0 (socket says 172.21.15.40:35656) 2023-10-16T08:39:46.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.326+0000 7fea23fff640 1 -- 172.21.15.40:0/380675563 learned_addr learned my addr 172.21.15.40:0/380675563 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:46.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.326+0000 7fea22ffd640 1 --2- 172.21.15.40:0/380675563 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea240cca60 0x7fea240b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.337 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.326+0000 7fea23fff640 1 -- 172.21.15.40:0/380675563 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea240cca60 msgr2=0x7fea240b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:46.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.326+0000 7fea23fff640 1 --2- 172.21.15.40:0/380675563 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea240cca60 0x7fea240b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.326+0000 7fea23fff640 1 -- 172.21.15.40:0/380675563 >> v1:172.21.15.86:6789/0 conn(0x7fea2415ad30 legacy=0x7fea24071080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.338 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.326+0000 7fea23fff640 1 -- 172.21.15.40:0/380675563 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea2406dbe0 con 0x7fea240cced0 2023-10-16T08:39:46.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.330+0000 7fea23fff640 1 --2- 172.21.15.40:0/380675563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea240cced0 0x7fea240b2d20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009e80 tx=0x7fea1c01c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78a88c11209a7f5 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:46.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.330+0000 7fea227fc640 1 -- 172.21.15.40:0/380675563 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0a57d0 con 0x7fea240cced0 2023-10-16T08:39:46.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.330+0000 7fea227fc640 1 -- 172.21.15.40:0/380675563 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c0a5930 con 0x7fea240cced0 2023-10-16T08:39:46.342 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.330+0000 7fea227fc640 1 -- 172.21.15.40:0/380675563 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0a5cc0 con 0x7fea240cced0 2023-10-16T08:39:46.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.330+0000 7fea29f62640 1 -- 172.21.15.40:0/380675563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea240cced0 msgr2=0x7fea240b2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.330+0000 7fea29f62640 1 --2- 172.21.15.40:0/380675563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea240cced0 0x7fea240b2d20 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fea1c009e80 tx=0x7fea1c01c9f0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.343 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.330+0000 7fea29f62640 1 -- 172.21.15.40:0/380675563 shutdown_connections 2023-10-16T08:39:46.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.330+0000 7fea29f62640 1 --2- 172.21.15.40:0/380675563 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea240cced0 0x7fea240b2d20 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.330+0000 7fea29f62640 1 --2- 172.21.15.40:0/380675563 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea240cca60 0x7fea240b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.330+0000 7fea29f62640 1 -- 172.21.15.40:0/380675563 >> 172.21.15.40:0/380675563 conn(0x7fea240b14a0 msgr2=0x7fea240b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.330+0000 7fea29f62640 1 -- 172.21.15.40:0/380675563 shutdown_connections 2023-10-16T08:39:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.330+0000 7fea29f62640 1 -- 172.21.15.40:0/380675563 wait complete. 2023-10-16T08:39:46.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea29f62640 1 Processor -- start 2023-10-16T08:39:46.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea29f62640 1 -- start start 2023-10-16T08:39:46.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea29f62640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea240cca60 0x7fea24154990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea29f62640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea240cced0 0x7fea24151f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea29f62640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea2415ad30 0x7fea240c61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea29f62640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea240b30d0 con 0x7fea2415ad30 2023-10-16T08:39:46.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea29f62640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea24154ed0 con 0x7fea240cca60 2023-10-16T08:39:46.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea29f62640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea24152480 con 0x7fea240cced0 2023-10-16T08:39:46.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea23fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea2415ad30 0x7fea240c61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea23fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea2415ad30 0x7fea240c61e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35664/0 (socket says 172.21.15.40:35664) 2023-10-16T08:39:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea23fff640 1 -- 172.21.15.40:0/4077553348 learned_addr learned my addr 172.21.15.40:0/4077553348 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea237fe640 1 --2- 172.21.15.40:0/4077553348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea240cca60 0x7fea24154990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea23fff640 1 -- 172.21.15.40:0/4077553348 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea240cced0 msgr2=0x7fea24151f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea22ffd640 1 --2- 172.21.15.40:0/4077553348 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea240cced0 0x7fea24151f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea23fff640 1 --2- 172.21.15.40:0/4077553348 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea240cced0 0x7fea24151f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea23fff640 1 -- 172.21.15.40:0/4077553348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea240cca60 msgr2=0x7fea24154990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea23fff640 1 --2- 172.21.15.40:0/4077553348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea240cca60 0x7fea24154990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea23fff640 1 -- 172.21.15.40:0/4077553348 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea1c009b30 con 0x7fea2415ad30 2023-10-16T08:39:46.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea23fff640 1 --2- 172.21.15.40:0/4077553348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea2415ad30 0x7fea240c61e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0af040 tx=0x7fea1c019790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:46.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea0bfff640 1 -- 172.21.15.40:0/4077553348 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0b8670 con 0x7fea2415ad30 2023-10-16T08:39:46.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea0bfff640 1 -- 172.21.15.40:0/4077553348 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea1c0b8810 con 0x7fea2415ad30 2023-10-16T08:39:46.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea29f62640 1 -- 172.21.15.40:0/4077553348 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea240c4000 con 0x7fea2415ad30 2023-10-16T08:39:46.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea0bfff640 1 -- 172.21.15.40:0/4077553348 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea1c0b8bc0 con 0x7fea2415ad30 2023-10-16T08:39:46.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea29f62640 1 -- 172.21.15.40:0/4077553348 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea240c5cc0 con 0x7fea2415ad30 2023-10-16T08:39:46.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea29f62640 1 -- 172.21.15.40:0/4077553348 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea240c5b20 con 0x7fea2415ad30 2023-10-16T08:39:46.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.334+0000 7fea0bfff640 1 -- 172.21.15.40:0/4077553348 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fea1c0b8d20 con 0x7fea2415ad30 2023-10-16T08:39:46.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.338+0000 7fea0bfff640 1 --2- 172.21.15.40:0/4077553348 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fea100664a0 0x7fea10068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.338+0000 7fea0bfff640 1 -- 172.21.15.40:0/4077553348 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fea1c11f2f0 con 0x7fea2415ad30 2023-10-16T08:39:46.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.338+0000 7fea237fe640 1 --2- 172.21.15.40:0/4077553348 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fea100664a0 0x7fea10068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.338+0000 7fea237fe640 1 --2- 172.21.15.40:0/4077553348 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fea100664a0 0x7fea10068960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fea2415d1a0 tx=0x7fea14009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:46.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.342+0000 7fea0bfff640 1 -- 172.21.15.40:0/4077553348 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fea240c5b20 con 0x7fea2415ad30 2023-10-16T08:39:46.501 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.494+0000 7fea29f62640 1 -- 172.21.15.40:0/4077553348 --> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fea240cc540 con 0x7fea100664a0 2023-10-16T08:39:46.502 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.494+0000 7fea0bfff640 1 -- 172.21.15.40:0/4077553348 <== mgr.4111 v2:172.21.15.86:6832/1188970899 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fea14001c80 con 0x7fea100664a0 2023-10-16T08:39:46.502 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 3 to deep-scrub 2023-10-16T08:39:46.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.498+0000 7fea29f62640 1 -- 172.21.15.40:0/4077553348 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fea100664a0 msgr2=0x7fea10068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.504 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.498+0000 7fea29f62640 1 --2- 172.21.15.40:0/4077553348 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fea100664a0 0x7fea10068960 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fea2415d1a0 tx=0x7fea14009040 comp rx=0 tx=0).stop 2023-10-16T08:39:46.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.498+0000 7fea29f62640 1 -- 172.21.15.40:0/4077553348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea2415ad30 msgr2=0x7fea240c61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.498+0000 7fea29f62640 1 --2- 172.21.15.40:0/4077553348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea2415ad30 0x7fea240c61e0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fea1c0af040 tx=0x7fea1c019790 comp rx=0 tx=0).stop 2023-10-16T08:39:46.505 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.498+0000 7fea29f62640 1 -- 172.21.15.40:0/4077553348 shutdown_connections 2023-10-16T08:39:46.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.498+0000 7fea29f62640 1 --2- 172.21.15.40:0/4077553348 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fea100664a0 0x7fea10068960 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.506 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.498+0000 7fea29f62640 1 --2- 172.21.15.40:0/4077553348 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fea2415ad30 0x7fea240c61e0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.498+0000 7fea29f62640 1 --2- 172.21.15.40:0/4077553348 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fea240cced0 0x7fea24151f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.498+0000 7fea29f62640 1 --2- 172.21.15.40:0/4077553348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea240cca60 0x7fea24154990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.498+0000 7fea29f62640 1 -- 172.21.15.40:0/4077553348 >> 172.21.15.40:0/4077553348 conn(0x7fea240b14a0 msgr2=0x7fea24155c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:46.507 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.498+0000 7fea29f62640 1 -- 172.21.15.40:0/4077553348 shutdown_connections 2023-10-16T08:39:46.508 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.498+0000 7fea29f62640 1 -- 172.21.15.40:0/4077553348 wait complete. 2023-10-16T08:39:46.520 INFO:tasks.ceph:Scrubbing osd.4 2023-10-16T08:39:46.520 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-16T08:39:46.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.638+0000 7fca32769640 1 Processor -- start 2023-10-16T08:39:46.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.642+0000 7fca32769640 1 -- start start 2023-10-16T08:39:46.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.642+0000 7fca32769640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fca2c0b41e0 0x7fca2c0cf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.642+0000 7fca32769640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca2c06c3c0 con 0x7fca2c15af20 2023-10-16T08:39:46.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.642+0000 7fca32769640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca2c0ae070 con 0x7fca2c15ab40 2023-10-16T08:39:46.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.642+0000 7fca32769640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca2c0b4750 con 0x7fca2c0b41e0 2023-10-16T08:39:46.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.642+0000 7fca30d06640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fca2c0b41e0 0x7fca2c0cf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.642+0000 7fca2b7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fca2c15af20 0x7fca2c0ca460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34068/0 (socket says 172.21.15.40:34068) 2023-10-16T08:39:46.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.642+0000 7fca2b7fe640 1 -- 172.21.15.40:0/2461193186 learned_addr learned my addr 172.21.15.40:0/2461193186 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:46.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.642+0000 7fca30d06640 1 -- 172.21.15.40:0/2461193186 >> v1:172.21.15.86:6789/0 conn(0x7fca2c15ab40 legacy=0x7fca2c070a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:46.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.642+0000 7fca30d06640 1 -- 172.21.15.40:0/2461193186 >> v1:172.21.15.40:6789/0 conn(0x7fca2c15af20 legacy=0x7fca2c0ca460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.642+0000 7fca30d06640 1 -- 172.21.15.40:0/2461193186 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca2c15e890 con 0x7fca2c0b41e0 2023-10-16T08:39:46.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.642+0000 7fca30d06640 1 --2- 172.21.15.40:0/2461193186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0b41e0 0x7fca2c0cf3d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fca240097c0 tx=0x7fca2401c2f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f8e851b6d51eb3f server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:46.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca2affd640 1 -- 172.21.15.40:0/2461193186 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca240ab600 con 0x7fca2c0b41e0 2023-10-16T08:39:46.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca2affd640 1 -- 172.21.15.40:0/2461193186 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca240a7050 con 0x7fca2c0b41e0 2023-10-16T08:39:46.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca2affd640 1 -- 172.21.15.40:0/2461193186 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca240ab980 con 0x7fca2c0b41e0 2023-10-16T08:39:46.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca32769640 1 -- 172.21.15.40:0/2461193186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0b41e0 msgr2=0x7fca2c0cf3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca32769640 1 --2- 172.21.15.40:0/2461193186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0b41e0 0x7fca2c0cf3d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fca240097c0 tx=0x7fca2401c2f0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca32769640 1 -- 172.21.15.40:0/2461193186 shutdown_connections 2023-10-16T08:39:46.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca32769640 1 --2- 172.21.15.40:0/2461193186 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0b41e0 0x7fca2c0cf3d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca32769640 1 -- 172.21.15.40:0/2461193186 >> 172.21.15.40:0/2461193186 conn(0x7fca2c0ce8b0 msgr2=0x7fca2c0cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:46.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca32769640 1 -- 172.21.15.40:0/2461193186 shutdown_connections 2023-10-16T08:39:46.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca32769640 1 -- 172.21.15.40:0/2461193186 wait complete. 2023-10-16T08:39:46.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca32769640 1 Processor -- start 2023-10-16T08:39:46.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca32769640 1 -- start start 2023-10-16T08:39:46.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca32769640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0b41e0 0x7fca2c155fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca32769640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca2c15ab40 0x7fca2c0c2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca32769640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca2c15af20 0x7fca2c0c2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca32769640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c0c74e0 con 0x7fca2c0b41e0 2023-10-16T08:39:46.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca32769640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c0c5600 con 0x7fca2c15ab40 2023-10-16T08:39:46.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca32769640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca2c0af0e0 con 0x7fca2c15af20 2023-10-16T08:39:46.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca2bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0b41e0 0x7fca2c155fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca2bfff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0b41e0 0x7fca2c155fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35688/0 (socket says 172.21.15.40:35688) 2023-10-16T08:39:46.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca2bfff640 1 -- 172.21.15.40:0/2957816607 learned_addr learned my addr 172.21.15.40:0/2957816607 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca30d06640 1 --2- 172.21.15.40:0/2957816607 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca2c15af20 0x7fca2c0c2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca2b7fe640 1 --2- 172.21.15.40:0/2957816607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca2c15ab40 0x7fca2c0c2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca2bfff640 1 -- 172.21.15.40:0/2957816607 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca2c15af20 msgr2=0x7fca2c0c2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca2bfff640 1 --2- 172.21.15.40:0/2957816607 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca2c15af20 0x7fca2c0c2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca2bfff640 1 -- 172.21.15.40:0/2957816607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca2c15ab40 msgr2=0x7fca2c0c2a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca2bfff640 1 --2- 172.21.15.40:0/2957816607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca2c15ab40 0x7fca2c0c2a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca2bfff640 1 -- 172.21.15.40:0/2957816607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca24009420 con 0x7fca2c0b41e0 2023-10-16T08:39:46.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca2b7fe640 1 --2- 172.21.15.40:0/2957816607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca2c15ab40 0x7fca2c0c2a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:46.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca30d06640 1 --2- 172.21.15.40:0/2957816607 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fca2c15af20 0x7fca2c0c2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:46.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca2bfff640 1 --2- 172.21.15.40:0/2957816607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0b41e0 0x7fca2c155fe0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00c7c0 tx=0x7fca1c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:46.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca28ff9640 1 -- 172.21.15.40:0/2957816607 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c00aa50 con 0x7fca2c0b41e0 2023-10-16T08:39:46.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca28ff9640 1 -- 172.21.15.40:0/2957816607 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c00abb0 con 0x7fca2c0b41e0 2023-10-16T08:39:46.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca28ff9640 1 -- 172.21.15.40:0/2957816607 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c012420 con 0x7fca2c0b41e0 2023-10-16T08:39:46.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca32769640 1 -- 172.21.15.40:0/2957816607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca2c0af3c0 con 0x7fca2c0b41e0 2023-10-16T08:39:46.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.646+0000 7fca32769640 1 -- 172.21.15.40:0/2957816607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca2c0c3b80 con 0x7fca2c0b41e0 2023-10-16T08:39:46.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.650+0000 7fca32769640 1 -- 172.21.15.40:0/2957816607 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fca2c15f130 con 0x7fca2c0b41e0 2023-10-16T08:39:46.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.650+0000 7fca28ff9640 1 -- 172.21.15.40:0/2957816607 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fca1c012580 con 0x7fca2c0b41e0 2023-10-16T08:39:46.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.650+0000 7fca28ff9640 1 --2- 172.21.15.40:0/2957816607 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fca180664a0 0x7fca18068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.650+0000 7fca28ff9640 1 -- 172.21.15.40:0/2957816607 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fca1c0848c0 con 0x7fca2c0b41e0 2023-10-16T08:39:46.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.650+0000 7fca28ff9640 1 --2- 172.21.15.40:0/2957816607 >> [v2:172.21.15.86:6808/150800249,v1:172.21.15.86:6809/150800249] conn(0x7fca1806e820 0x7fca18070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.650+0000 7fca28ff9640 1 -- 172.21.15.40:0/2957816607 --> [v2:172.21.15.86:6808/150800249,v1:172.21.15.86:6809/150800249] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fca18071390 con 0x7fca1806e820 2023-10-16T08:39:46.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.650+0000 7fca28ff9640 1 -- 172.21.15.40:0/2957816607 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fca1c084c30 con 0x7fca2c0b41e0 2023-10-16T08:39:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.650+0000 7fca2b7fe640 1 --2- 172.21.15.40:0/2957816607 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fca180664a0 0x7fca18068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.650+0000 7fca30d06640 1 --2- 172.21.15.40:0/2957816607 >> [v2:172.21.15.86:6808/150800249,v1:172.21.15.86:6809/150800249] conn(0x7fca1806e820 0x7fca18070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.650+0000 7fca30d06640 1 --2- 172.21.15.40:0/2957816607 >> [v2:172.21.15.86:6808/150800249,v1:172.21.15.86:6809/150800249] conn(0x7fca1806e820 0x7fca18070cc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.650+0000 7fca2b7fe640 1 --2- 172.21.15.40:0/2957816607 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fca180664a0 0x7fca18068960 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fca2c154d80 tx=0x7fca20009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:46.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.654+0000 7fca28ff9640 1 -- 172.21.15.40:0/2957816607 <== osd.4 v2:172.21.15.86:6808/150800249 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fca18071390 con 0x7fca1806e820 2023-10-16T08:39:46.672 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.666+0000 7fca32769640 1 -- 172.21.15.40:0/2957816607 --> [v2:172.21.15.86:6808/150800249,v1:172.21.15.86:6809/150800249] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fca2c0ca930 con 0x7fca1806e820 2023-10-16T08:39:46.673 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.666+0000 7fca28ff9640 1 -- 172.21.15.40:0/2957816607 <== osd.4 v2:172.21.15.86:6808/150800249 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fca2c0ca930 con 0x7fca1806e820 2023-10-16T08:39:46.674 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-16T08:39:46.674 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T08:39:46.675 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-16T08:39:46.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.666+0000 7fca32769640 1 -- 172.21.15.40:0/2957816607 >> [v2:172.21.15.86:6808/150800249,v1:172.21.15.86:6809/150800249] conn(0x7fca1806e820 msgr2=0x7fca18070cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.666+0000 7fca32769640 1 --2- 172.21.15.40:0/2957816607 >> [v2:172.21.15.86:6808/150800249,v1:172.21.15.86:6809/150800249] conn(0x7fca1806e820 0x7fca18070cc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.676 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.666+0000 7fca32769640 1 -- 172.21.15.40:0/2957816607 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fca180664a0 msgr2=0x7fca18068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.666+0000 7fca32769640 1 --2- 172.21.15.40:0/2957816607 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fca180664a0 0x7fca18068960 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fca2c154d80 tx=0x7fca20009040 comp rx=0 tx=0).stop 2023-10-16T08:39:46.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.666+0000 7fca32769640 1 -- 172.21.15.40:0/2957816607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0b41e0 msgr2=0x7fca2c155fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.666+0000 7fca32769640 1 --2- 172.21.15.40:0/2957816607 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fca2c0b41e0 0x7fca2c155fe0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fca1c00c7c0 tx=0x7fca1c00cc80 comp rx=0 tx=0).stop 2023-10-16T08:39:46.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.666+0000 7fca30d06640 1 -- 172.21.15.40:0/2957816607 reap_dead start 2023-10-16T08:39:46.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.666+0000 7fca32769640 1 -- 172.21.15.40:0/2957816607 shutdown_connections 2023-10-16T08:39:46.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.666+0000 7fca32769640 1 -- 172.21.15.40:0/2957816607 >> 172.21.15.40:0/2957816607 conn(0x7fca2c0ce8b0 msgr2=0x7fca2c0ae370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:46.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.666+0000 7fca32769640 1 -- 172.21.15.40:0/2957816607 shutdown_connections 2023-10-16T08:39:46.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.666+0000 7fca32769640 1 -- 172.21.15.40:0/2957816607 wait complete. 2023-10-16T08:39:46.691 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-16T08:39:46.810 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.802+0000 7fd85b94c640 1 Processor -- start 2023-10-16T08:39:46.811 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.802+0000 7fd85b94c640 1 -- start start 2023-10-16T08:39:46.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.806+0000 7fd85b94c640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fd8540af740 0x7fd8540b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.806+0000 7fd85b94c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8540cf770 0x7fd8540adb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.813 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.806+0000 7fd85b94c640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8540b1ec0 con 0x7fd8540d15c0 2023-10-16T08:39:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.806+0000 7fd85b94c640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8540b24c0 con 0x7fd8540cf770 2023-10-16T08:39:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.806+0000 7fd85b94c640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8540b10a0 con 0x7fd8540af740 2023-10-16T08:39:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.806+0000 7fd859ee9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8540cf770 0x7fd8540adb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.814 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.806+0000 7fd859ee9640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fd8540cf770 0x7fd8540adb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35696/0 (socket says 172.21.15.40:35696) 2023-10-16T08:39:46.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.806+0000 7fd859ee9640 1 -- 172.21.15.40:0/4105981233 learned_addr learned my addr 172.21.15.40:0/4105981233 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:46.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.806+0000 7fd8596e8640 1 --2- 172.21.15.40:0/4105981233 >> v2:172.21.15.86:3300/0 conn(0x7fd8540af740 0x7fd8540b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.815 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.806+0000 7fd859ee9640 1 -- 172.21.15.40:0/4105981233 >> v2:172.21.15.86:3300/0 conn(0x7fd8540af740 msgr2=0x7fd8540b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:46.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.806+0000 7fd859ee9640 1 --2- 172.21.15.40:0/4105981233 >> v2:172.21.15.86:3300/0 conn(0x7fd8540af740 0x7fd8540b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.816 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.806+0000 7fd859ee9640 1 -- 172.21.15.40:0/4105981233 >> v1:172.21.15.86:6789/0 conn(0x7fd8540d15c0 legacy=0x7fd8540cf060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:46.817 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.806+0000 7fd859ee9640 1 -- 172.21.15.40:0/4105981233 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8540b1340 con 0x7fd8540cf770 2023-10-16T08:39:46.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.806+0000 7fd859ee9640 1 --2- 172.21.15.40:0/4105981233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8540cf770 0x7fd8540adb50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd854070c10 tx=0x7fd84c01c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2a5e879ca25eaa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:46.820 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.806+0000 7fd84bfff640 1 -- 172.21.15.40:0/4105981233 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c0a5820 con 0x7fd8540cf770 2023-10-16T08:39:46.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.806+0000 7fd84bfff640 1 -- 172.21.15.40:0/4105981233 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd84c0a5980 con 0x7fd8540cf770 2023-10-16T08:39:46.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.806+0000 7fd84bfff640 1 -- 172.21.15.40:0/4105981233 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd84c0a5d10 con 0x7fd8540cf770 2023-10-16T08:39:46.821 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd85b94c640 1 -- 172.21.15.40:0/4105981233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8540cf770 msgr2=0x7fd8540adb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd85b94c640 1 --2- 172.21.15.40:0/4105981233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8540cf770 0x7fd8540adb50 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd854070c10 tx=0x7fd84c01c960 comp rx=0 tx=0).stop 2023-10-16T08:39:46.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd85b94c640 1 -- 172.21.15.40:0/4105981233 shutdown_connections 2023-10-16T08:39:46.822 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd85b94c640 1 --2- 172.21.15.40:0/4105981233 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8540cf770 0x7fd8540adb50 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd85b94c640 1 --2- 172.21.15.40:0/4105981233 >> v2:172.21.15.86:3300/0 conn(0x7fd8540af740 0x7fd8540b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd85b94c640 1 -- 172.21.15.40:0/4105981233 >> 172.21.15.40:0/4105981233 conn(0x7fd854099fa0 msgr2=0x7fd8540d0c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:46.823 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd85b94c640 1 -- 172.21.15.40:0/4105981233 shutdown_connections 2023-10-16T08:39:46.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd85b94c640 1 -- 172.21.15.40:0/4105981233 wait complete. 2023-10-16T08:39:46.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd85b94c640 1 Processor -- start 2023-10-16T08:39:46.824 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd85b94c640 1 -- start start 2023-10-16T08:39:46.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd85b94c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8540af740 0x7fd8540c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd85b94c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8540cf770 0x7fd85415d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd85b94c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8540d15c0 0x7fd85415d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.825 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd85b94c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85409cb20 con 0x7fd8540af740 2023-10-16T08:39:46.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd85b94c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85409d980 con 0x7fd8540cf770 2023-10-16T08:39:46.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd85b94c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85409cd70 con 0x7fd8540d15c0 2023-10-16T08:39:46.826 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd8596e8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8540af740 0x7fd8540c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd8596e8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8540af740 0x7fd8540c7ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35712/0 (socket says 172.21.15.40:35712) 2023-10-16T08:39:46.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd8596e8640 1 -- 172.21.15.40:0/2477433697 learned_addr learned my addr 172.21.15.40:0/2477433697 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:46.827 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd858ee7640 1 --2- 172.21.15.40:0/2477433697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8540cf770 0x7fd85415d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd859ee9640 1 --2- 172.21.15.40:0/2477433697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8540d15c0 0x7fd85415d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd8596e8640 1 -- 172.21.15.40:0/2477433697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8540d15c0 msgr2=0x7fd85415d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.828 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd8596e8640 1 --2- 172.21.15.40:0/2477433697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8540d15c0 0x7fd85415d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd8596e8640 1 -- 172.21.15.40:0/2477433697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8540cf770 msgr2=0x7fd85415d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd8596e8640 1 --2- 172.21.15.40:0/2477433697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8540cf770 0x7fd85415d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.829 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd8596e8640 1 -- 172.21.15.40:0/2477433697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd84c009b30 con 0x7fd8540af740 2023-10-16T08:39:46.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd859ee9640 1 --2- 172.21.15.40:0/2477433697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8540d15c0 0x7fd85415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:46.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd8596e8640 1 --2- 172.21.15.40:0/2477433697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8540af740 0x7fd8540c7ce0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd85000d830 tx=0x7fd85000dcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:46.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd849ffb640 1 -- 172.21.15.40:0/2477433697 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85000b6c0 con 0x7fd8540af740 2023-10-16T08:39:46.830 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd849ffb640 1 -- 172.21.15.40:0/2477433697 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85000b820 con 0x7fd8540af740 2023-10-16T08:39:46.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd85b94c640 1 -- 172.21.15.40:0/2477433697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd85409d050 con 0x7fd8540af740 2023-10-16T08:39:46.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd849ffb640 1 -- 172.21.15.40:0/2477433697 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85000bb90 con 0x7fd8540af740 2023-10-16T08:39:46.831 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.810+0000 7fd85b94c640 1 -- 172.21.15.40:0/2477433697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd85409d610 con 0x7fd8540af740 2023-10-16T08:39:46.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.814+0000 7fd85b94c640 1 -- 172.21.15.40:0/2477433697 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd85404e9c0 con 0x7fd8540af740 2023-10-16T08:39:46.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.814+0000 7fd849ffb640 1 -- 172.21.15.40:0/2477433697 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fd85000bcf0 con 0x7fd8540af740 2023-10-16T08:39:46.832 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.814+0000 7fd849ffb640 1 --2- 172.21.15.40:0/2477433697 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd8380664a0 0x7fd838068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:46.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.814+0000 7fd849ffb640 1 -- 172.21.15.40:0/2477433697 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fd850084420 con 0x7fd8540af740 2023-10-16T08:39:46.833 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.814+0000 7fd858ee7640 1 --2- 172.21.15.40:0/2477433697 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd8380664a0 0x7fd838068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:46.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.814+0000 7fd858ee7640 1 --2- 172.21.15.40:0/2477433697 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd8380664a0 0x7fd838068960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd854158880 tx=0x7fd844008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:46.834 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.818+0000 7fd849ffb640 1 -- 172.21.15.40:0/2477433697 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fd850051810 con 0x7fd8540af740 2023-10-16T08:39:46.980 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.974+0000 7fd85b94c640 1 -- 172.21.15.40:0/2477433697 --> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd8540b1be0 con 0x7fd8380664a0 2023-10-16T08:39:46.981 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.974+0000 7fd849ffb640 1 -- 172.21.15.40:0/2477433697 <== mgr.4111 v2:172.21.15.86:6832/1188970899 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd84400bdc0 con 0x7fd8380664a0 2023-10-16T08:39:46.982 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 4 to deep-scrub 2023-10-16T08:39:46.984 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.978+0000 7fd85b94c640 1 -- 172.21.15.40:0/2477433697 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd8380664a0 msgr2=0x7fd838068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.978+0000 7fd85b94c640 1 --2- 172.21.15.40:0/2477433697 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd8380664a0 0x7fd838068960 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fd854158880 tx=0x7fd844008040 comp rx=0 tx=0).stop 2023-10-16T08:39:46.985 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.978+0000 7fd85b94c640 1 -- 172.21.15.40:0/2477433697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8540af740 msgr2=0x7fd8540c7ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:46.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.978+0000 7fd85b94c640 1 --2- 172.21.15.40:0/2477433697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8540af740 0x7fd8540c7ce0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fd85000d830 tx=0x7fd85000dcf0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.986 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.978+0000 7fd85b94c640 1 -- 172.21.15.40:0/2477433697 shutdown_connections 2023-10-16T08:39:46.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.978+0000 7fd85b94c640 1 --2- 172.21.15.40:0/2477433697 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fd8540d15c0 0x7fd85415d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.987 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.978+0000 7fd85b94c640 1 --2- 172.21.15.40:0/2477433697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd8540cf770 0x7fd85415d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.978+0000 7fd85b94c640 1 --2- 172.21.15.40:0/2477433697 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fd8380664a0 0x7fd838068960 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.988 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.978+0000 7fd85b94c640 1 --2- 172.21.15.40:0/2477433697 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fd8540af740 0x7fd8540c7ce0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:46.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.978+0000 7fd85b94c640 1 -- 172.21.15.40:0/2477433697 >> 172.21.15.40:0/2477433697 conn(0x7fd854099fa0 msgr2=0x7fd8540aff30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:46.989 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.978+0000 7fd85b94c640 1 -- 172.21.15.40:0/2477433697 shutdown_connections 2023-10-16T08:39:46.990 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:46.978+0000 7fd85b94c640 1 -- 172.21.15.40:0/2477433697 wait complete. 2023-10-16T08:39:47.000 INFO:tasks.ceph:Scrubbing osd.5 2023-10-16T08:39:47.001 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-16T08:39:47.133 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.126+0000 7fc8ab730640 1 Processor -- start 2023-10-16T08:39:47.134 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.126+0000 7fc8ab730640 1 -- start start 2023-10-16T08:39:47.135 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.126+0000 7fc8ab730640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8a40ce540 0x7fc8a40cd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.126+0000 7fc8ab730640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc8a40afb90 0x7fc8a40cfbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.126+0000 7fc8ab730640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8a40cd2e0 con 0x7fc8a40d00f0 2023-10-16T08:39:47.136 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.126+0000 7fc8ab730640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8a40cc6d0 con 0x7fc8a40afb90 2023-10-16T08:39:47.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.126+0000 7fc8ab730640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a40cc9b0 con 0x7fc8a40ce540 2023-10-16T08:39:47.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.126+0000 7fc8a8ccb640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc8a40afb90 0x7fc8a40cfbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.137 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.126+0000 7fc8a9ccd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fc8a40d00f0 0x7fc8a40ce920 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34080/0 (socket says 172.21.15.40:34080) 2023-10-16T08:39:47.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.130+0000 7fc8a8ccb640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fc8a40afb90 0x7fc8a40cfbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35716/0 (socket says 172.21.15.40:35716) 2023-10-16T08:39:47.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.130+0000 7fc8a9ccd640 1 -- 172.21.15.40:0/2916240851 learned_addr learned my addr 172.21.15.40:0/2916240851 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:47.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.130+0000 7fc8a94cc640 1 --2- 172.21.15.40:0/2916240851 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8a40ce540 0x7fc8a40cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.138 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.130+0000 7fc8a8ccb640 1 -- 172.21.15.40:0/2916240851 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8a40ce540 msgr2=0x7fc8a40cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:47.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.130+0000 7fc8a8ccb640 1 --2- 172.21.15.40:0/2916240851 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8a40ce540 0x7fc8a40cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.130+0000 7fc8a8ccb640 1 -- 172.21.15.40:0/2916240851 >> v1:172.21.15.40:6789/0 conn(0x7fc8a40d00f0 legacy=0x7fc8a40ce920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.139 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.130+0000 7fc8a8ccb640 1 -- 172.21.15.40:0/2916240851 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a40ccc90 con 0x7fc8a40afb90 2023-10-16T08:39:47.142 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.130+0000 7fc8a8ccb640 1 --2- 172.21.15.40:0/2916240851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8a40afb90 0x7fc8a40cfbb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00097c0 tx=0x7fc8a001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a50c29c6eefb9fa server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:47.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.130+0000 7fc897fff640 1 -- 172.21.15.40:0/2916240851 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a00ab620 con 0x7fc8a40afb90 2023-10-16T08:39:47.143 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.130+0000 7fc897fff640 1 -- 172.21.15.40:0/2916240851 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8a00ab780 con 0x7fc8a40afb90 2023-10-16T08:39:47.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.130+0000 7fc897fff640 1 -- 172.21.15.40:0/2916240851 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8a00abb10 con 0x7fc8a40afb90 2023-10-16T08:39:47.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.130+0000 7fc8ab730640 1 -- 172.21.15.40:0/2916240851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8a40afb90 msgr2=0x7fc8a40cfbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.130+0000 7fc8ab730640 1 --2- 172.21.15.40:0/2916240851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8a40afb90 0x7fc8a40cfbb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00097c0 tx=0x7fc8a001c480 comp rx=0 tx=0).stop 2023-10-16T08:39:47.144 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.130+0000 7fc8ab730640 1 -- 172.21.15.40:0/2916240851 shutdown_connections 2023-10-16T08:39:47.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.130+0000 7fc8ab730640 1 --2- 172.21.15.40:0/2916240851 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8a40afb90 0x7fc8a40cfbb0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.130+0000 7fc8ab730640 1 --2- 172.21.15.40:0/2916240851 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8a40ce540 0x7fc8a40cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.145 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.130+0000 7fc8ab730640 1 -- 172.21.15.40:0/2916240851 >> 172.21.15.40:0/2916240851 conn(0x7fc8a4065e80 msgr2=0x7fc8a4066290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:47.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.130+0000 7fc8ab730640 1 -- 172.21.15.40:0/2916240851 shutdown_connections 2023-10-16T08:39:47.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.130+0000 7fc8ab730640 1 -- 172.21.15.40:0/2916240851 wait complete. 2023-10-16T08:39:47.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8ab730640 1 Processor -- start 2023-10-16T08:39:47.146 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8ab730640 1 -- start start 2023-10-16T08:39:47.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8ab730640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8a40afb90 0x7fc8a414a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8ab730640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8a40ce540 0x7fc8a40c61a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.147 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8ab730640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a40d00f0 0x7fc8a4152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8ab730640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a414a890 con 0x7fc8a40afb90 2023-10-16T08:39:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8ab730640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a4153a10 con 0x7fc8a40d00f0 2023-10-16T08:39:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8ab730640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8a4153cf0 con 0x7fc8a40ce540 2023-10-16T08:39:47.148 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8a94cc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8a40afb90 0x7fc8a414a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8a94cc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8a40afb90 0x7fc8a414a350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35730/0 (socket says 172.21.15.40:35730) 2023-10-16T08:39:47.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8a94cc640 1 -- 172.21.15.40:0/3331513630 learned_addr learned my addr 172.21.15.40:0/3331513630 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:47.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8a8ccb640 1 --2- 172.21.15.40:0/3331513630 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8a40ce540 0x7fc8a40c61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.149 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8a9ccd640 1 --2- 172.21.15.40:0/3331513630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a40d00f0 0x7fc8a4152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8a94cc640 1 -- 172.21.15.40:0/3331513630 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8a40ce540 msgr2=0x7fc8a40c61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8a94cc640 1 --2- 172.21.15.40:0/3331513630 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fc8a40ce540 0x7fc8a40c61a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8a94cc640 1 -- 172.21.15.40:0/3331513630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a40d00f0 msgr2=0x7fc8a4152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.150 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8a94cc640 1 --2- 172.21.15.40:0/3331513630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc8a40d00f0 0x7fc8a4152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8a94cc640 1 -- 172.21.15.40:0/3331513630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8a0009420 con 0x7fc8a40afb90 2023-10-16T08:39:47.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8a94cc640 1 --2- 172.21.15.40:0/3331513630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8a40afb90 0x7fc8a414a350 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc89800d5f0 tx=0x7fc89800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:47.151 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc895ffb640 1 -- 172.21.15.40:0/3331513630 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8980125f0 con 0x7fc8a40afb90 2023-10-16T08:39:47.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc895ffb640 1 -- 172.21.15.40:0/3331513630 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc898012750 con 0x7fc8a40afb90 2023-10-16T08:39:47.152 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc895ffb640 1 -- 172.21.15.40:0/3331513630 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc898012ae0 con 0x7fc8a40afb90 2023-10-16T08:39:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8ab730640 1 -- 172.21.15.40:0/3331513630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8a40cde30 con 0x7fc8a40afb90 2023-10-16T08:39:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8ab730640 1 -- 172.21.15.40:0/3331513630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8a40c59a0 con 0x7fc8a40afb90 2023-10-16T08:39:47.153 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.134+0000 7fc8ab730640 1 -- 172.21.15.40:0/3331513630 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc8a40cdcb0 con 0x7fc8a40afb90 2023-10-16T08:39:47.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.138+0000 7fc895ffb640 1 -- 172.21.15.40:0/3331513630 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fc898012c40 con 0x7fc8a40afb90 2023-10-16T08:39:47.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.138+0000 7fc895ffb640 1 --2- 172.21.15.40:0/3331513630 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fc8880664a0 0x7fc888068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.155 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.138+0000 7fc895ffb640 1 -- 172.21.15.40:0/3331513630 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fc898084ef0 con 0x7fc8a40afb90 2023-10-16T08:39:47.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.138+0000 7fc895ffb640 1 --2- 172.21.15.40:0/3331513630 >> [v2:172.21.15.86:6824/751032064,v1:172.21.15.86:6825/751032064] conn(0x7fc88806e820 0x7fc888070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.156 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.138+0000 7fc895ffb640 1 -- 172.21.15.40:0/3331513630 --> [v2:172.21.15.86:6824/751032064,v1:172.21.15.86:6825/751032064] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc888071390 con 0x7fc88806e820 2023-10-16T08:39:47.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.138+0000 7fc895ffb640 1 -- 172.21.15.40:0/3331513630 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc898085260 con 0x7fc8a40afb90 2023-10-16T08:39:47.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.138+0000 7fc8a8ccb640 1 --2- 172.21.15.40:0/3331513630 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fc8880664a0 0x7fc888068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.138+0000 7fc8a9ccd640 1 --2- 172.21.15.40:0/3331513630 >> [v2:172.21.15.86:6824/751032064,v1:172.21.15.86:6825/751032064] conn(0x7fc88806e820 0x7fc888070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.157 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.138+0000 7fc8a8ccb640 1 --2- 172.21.15.40:0/3331513630 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fc8880664a0 0x7fc888068960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00af000 tx=0x7fc8a00a7000 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:47.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.138+0000 7fc8a9ccd640 1 --2- 172.21.15.40:0/3331513630 >> [v2:172.21.15.86:6824/751032064,v1:172.21.15.86:6825/751032064] conn(0x7fc88806e820 0x7fc888070cc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:47.158 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.138+0000 7fc895ffb640 1 -- 172.21.15.40:0/3331513630 <== osd.5 v2:172.21.15.86:6824/751032064 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7fc888071390 con 0x7fc88806e820 2023-10-16T08:39:47.159 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.154+0000 7fc8ab730640 1 -- 172.21.15.40:0/3331513630 --> [v2:172.21.15.86:6824/751032064,v1:172.21.15.86:6825/751032064] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7fc8a40d0500 con 0x7fc88806e820 2023-10-16T08:39:47.160 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.154+0000 7fc895ffb640 1 -- 172.21.15.40:0/3331513630 <== osd.5 v2:172.21.15.86:6824/751032064 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7fc89c0070f0 con 0x7fc88806e820 2023-10-16T08:39:47.161 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-16T08:39:47.161 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T08:39:47.161 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-16T08:39:47.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.154+0000 7fc8ab730640 1 -- 172.21.15.40:0/3331513630 >> [v2:172.21.15.86:6824/751032064,v1:172.21.15.86:6825/751032064] conn(0x7fc88806e820 msgr2=0x7fc888070cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.154+0000 7fc8ab730640 1 --2- 172.21.15.40:0/3331513630 >> [v2:172.21.15.86:6824/751032064,v1:172.21.15.86:6825/751032064] conn(0x7fc88806e820 0x7fc888070cc0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.154+0000 7fc8ab730640 1 -- 172.21.15.40:0/3331513630 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fc8880664a0 msgr2=0x7fc888068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.163 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.154+0000 7fc8ab730640 1 --2- 172.21.15.40:0/3331513630 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fc8880664a0 0x7fc888068960 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fc8a00af000 tx=0x7fc8a00a7000 comp rx=0 tx=0).stop 2023-10-16T08:39:47.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.154+0000 7fc8ab730640 1 -- 172.21.15.40:0/3331513630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8a40afb90 msgr2=0x7fc8a414a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.154+0000 7fc8ab730640 1 --2- 172.21.15.40:0/3331513630 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fc8a40afb90 0x7fc8a414a350 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc89800d5f0 tx=0x7fc89800dab0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.164 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.154+0000 7fc8a9ccd640 1 -- 172.21.15.40:0/3331513630 reap_dead start 2023-10-16T08:39:47.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.154+0000 7fc8ab730640 1 -- 172.21.15.40:0/3331513630 shutdown_connections 2023-10-16T08:39:47.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.154+0000 7fc8ab730640 1 -- 172.21.15.40:0/3331513630 >> 172.21.15.40:0/3331513630 conn(0x7fc8a4065e80 msgr2=0x7fc8a40b03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:47.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.154+0000 7fc8ab730640 1 -- 172.21.15.40:0/3331513630 shutdown_connections 2023-10-16T08:39:47.165 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.154+0000 7fc8ab730640 1 -- 172.21.15.40:0/3331513630 wait complete. 2023-10-16T08:39:47.177 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-16T08:39:47.289 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.282+0000 7fa7432cc640 1 Processor -- start 2023-10-16T08:39:47.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.282+0000 7fa7432cc640 1 -- start start 2023-10-16T08:39:47.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.282+0000 7fa7432cc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0ad9e0 0x7fa73c159ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.282+0000 7fa7432cc640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa73c15dca0 0x7fa73c0cca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.282+0000 7fa7432cc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa73c15d530 con 0x7fa73c06c650 2023-10-16T08:39:47.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.282+0000 7fa7432cc640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa73c0ca350 con 0x7fa73c15dca0 2023-10-16T08:39:47.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.282+0000 7fa7432cc640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c15a7d0 con 0x7fa73c0ad9e0 2023-10-16T08:39:47.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.282+0000 7fa741068640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa73c06c650 0x7fa73c06ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34086/0 (socket says 172.21.15.40:34086) 2023-10-16T08:39:47.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.282+0000 7fa741068640 1 -- 172.21.15.40:0/1507554380 learned_addr learned my addr 172.21.15.40:0/1507554380 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:47.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa740867640 1 --2- 172.21.15.40:0/1507554380 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0ad9e0 0x7fa73c159ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa741869640 1 --2- 172.21.15.40:0/1507554380 >> v2:172.21.15.86:3300/0 conn(0x7fa73c15dca0 0x7fa73c0cca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa733fff640 1 -- 172.21.15.40:0/1507554380 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582618723 0 0) 0x7fa73c15d530 con 0x7fa73c06c650 2023-10-16T08:39:47.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa733fff640 1 -- 172.21.15.40:0/1507554380 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa728003530 con 0x7fa73c06c650 2023-10-16T08:39:47.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa733fff640 1 -- 172.21.15.40:0/1507554380 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1280517199 0 0) 0x7fa728003530 con 0x7fa73c06c650 2023-10-16T08:39:47.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa733fff640 1 -- 172.21.15.40:0/1507554380 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0ad9e0 msgr2=0x7fa73c159ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:47.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa733fff640 1 --2- 172.21.15.40:0/1507554380 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0ad9e0 0x7fa73c159ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa733fff640 1 -- 172.21.15.40:0/1507554380 >> v2:172.21.15.86:3300/0 conn(0x7fa73c15dca0 msgr2=0x7fa73c0cca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:47.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa733fff640 1 --2- 172.21.15.40:0/1507554380 >> v2:172.21.15.86:3300/0 conn(0x7fa73c15dca0 0x7fa73c0cca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa733fff640 1 -- 172.21.15.40:0/1507554380 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c15aab0 con 0x7fa73c06c650 2023-10-16T08:39:47.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa733fff640 1 -- 172.21.15.40:0/1507554380 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7fa738001680 con 0x7fa73c06c650 2023-10-16T08:39:47.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa733fff640 1 -- 172.21.15.40:0/1507554380 >> v1:172.21.15.40:6789/0 conn(0x7fa73c06c650 legacy=0x7fa73c06ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa733fff640 1 --2- 172.21.15.40:0/1507554380 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0ad9e0 0x7fa728003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa733fff640 1 --2- 172.21.15.40:0/1507554380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa728003f50 0x7fa7280047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa733fff640 1 --2- 172.21.15.40:0/1507554380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa728004d10 0x7fa728005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa733fff640 1 -- 172.21.15.40:0/1507554380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c0ca350 con 0x7fa728003f50 2023-10-16T08:39:47.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa733fff640 1 -- 172.21.15.40:0/1507554380 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c15a7d0 con 0x7fa728004d10 2023-10-16T08:39:47.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa733fff640 1 -- 172.21.15.40:0/1507554380 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72800bac0 con 0x7fa73c0ad9e0 2023-10-16T08:39:47.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa740867640 1 --2- 172.21.15.40:0/1507554380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa728003f50 0x7fa7280047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa741068640 1 --2- 172.21.15.40:0/1507554380 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0ad9e0 0x7fa728003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa741869640 1 --2- 172.21.15.40:0/1507554380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa728004d10 0x7fa728005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa740867640 1 -- 172.21.15.40:0/1507554380 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0ad9e0 msgr2=0x7fa728003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:47.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa740867640 1 --2- 172.21.15.40:0/1507554380 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0ad9e0 0x7fa728003980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa740867640 1 -- 172.21.15.40:0/1507554380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa728004d10 msgr2=0x7fa728005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:47.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa740867640 1 --2- 172.21.15.40:0/1507554380 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa728004d10 0x7fa728005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa740867640 1 -- 172.21.15.40:0/1507554380 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa728003530 con 0x7fa728003f50 2023-10-16T08:39:47.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa740867640 1 --2- 172.21.15.40:0/1507554380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa728003f50 0x7fa7280047d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009ae0 tx=0x7fa72c01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11e43c8576f22318 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:47.305 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa733fff640 1 -- 172.21.15.40:0/1507554380 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c0a6510 con 0x7fa728003f50 2023-10-16T08:39:47.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa733fff640 1 -- 172.21.15.40:0/1507554380 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa72c0a6670 con 0x7fa728003f50 2023-10-16T08:39:47.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa733fff640 1 -- 172.21.15.40:0/1507554380 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa72c0a6a20 con 0x7fa728003f50 2023-10-16T08:39:47.306 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa7432cc640 1 -- 172.21.15.40:0/1507554380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa728003f50 msgr2=0x7fa7280047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa7432cc640 1 --2- 172.21.15.40:0/1507554380 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa728003f50 0x7fa7280047d0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fa72c009ae0 tx=0x7fa72c01ca20 comp rx=0 tx=0).stop 2023-10-16T08:39:47.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.286+0000 7fa741869640 1 -- 172.21.15.40:0/1507554380 reap_dead start 2023-10-16T08:39:47.307 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa7432cc640 1 -- 172.21.15.40:0/1507554380 shutdown_connections 2023-10-16T08:39:47.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa7432cc640 1 -- 172.21.15.40:0/1507554380 >> 172.21.15.40:0/1507554380 conn(0x7fa73c0cb400 msgr2=0x7fa73c0cb810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:47.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa7432cc640 1 -- 172.21.15.40:0/1507554380 shutdown_connections 2023-10-16T08:39:47.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa7432cc640 1 -- 172.21.15.40:0/1507554380 wait complete. 2023-10-16T08:39:47.308 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa7432cc640 1 Processor -- start 2023-10-16T08:39:47.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa7432cc640 1 -- start start 2023-10-16T08:39:47.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa7432cc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa728003f50 0x7fa73c153350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa7432cc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0c85a0 0x7fa73c0acba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.309 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa7432cc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa73c0c24b0 0x7fa73c0c2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa7432cc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c0c8b10 con 0x7fa73c0c24b0 2023-10-16T08:39:47.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa7432cc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c0c37f0 con 0x7fa728003f50 2023-10-16T08:39:47.310 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa7432cc640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa73c0c3ad0 con 0x7fa73c0c85a0 2023-10-16T08:39:47.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa741869640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa73c0c24b0 0x7fa73c0c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa741869640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa73c0c24b0 0x7fa73c0c2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35744/0 (socket says 172.21.15.40:35744) 2023-10-16T08:39:47.311 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa741869640 1 -- 172.21.15.40:0/3778750001 learned_addr learned my addr 172.21.15.40:0/3778750001 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:47.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa741068640 1 --2- 172.21.15.40:0/3778750001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa728003f50 0x7fa73c153350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa740867640 1 --2- 172.21.15.40:0/3778750001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0c85a0 0x7fa73c0acba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.312 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa741869640 1 -- 172.21.15.40:0/3778750001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0c85a0 msgr2=0x7fa73c0acba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa741869640 1 --2- 172.21.15.40:0/3778750001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0c85a0 0x7fa73c0acba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa741869640 1 -- 172.21.15.40:0/3778750001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa728003f50 msgr2=0x7fa73c153350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.313 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa741869640 1 --2- 172.21.15.40:0/3778750001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa728003f50 0x7fa73c153350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa741869640 1 -- 172.21.15.40:0/3778750001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa73c0c3db0 con 0x7fa73c0c24b0 2023-10-16T08:39:47.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa740867640 1 --2- 172.21.15.40:0/3778750001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0c85a0 0x7fa73c0acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:47.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa741068640 1 --2- 172.21.15.40:0/3778750001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa728003f50 0x7fa73c153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-16T08:39:47.314 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa741869640 1 --2- 172.21.15.40:0/3778750001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa73c0c24b0 0x7fa73c0c2960 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0b48b0 tx=0x7fa73400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:47.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa731ffb640 1 -- 172.21.15.40:0/3778750001 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa73400e040 con 0x7fa73c0c24b0 2023-10-16T08:39:47.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa731ffb640 1 -- 172.21.15.40:0/3778750001 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa73400f040 con 0x7fa73c0c24b0 2023-10-16T08:39:47.315 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa7432cc640 1 -- 172.21.15.40:0/3778750001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73c1f5a50 con 0x7fa73c0c24b0 2023-10-16T08:39:47.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa731ffb640 1 -- 172.21.15.40:0/3778750001 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa734015430 con 0x7fa73c0c24b0 2023-10-16T08:39:47.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.290+0000 7fa7432cc640 1 -- 172.21.15.40:0/3778750001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73c0ad0e0 con 0x7fa73c0c24b0 2023-10-16T08:39:47.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.294+0000 7fa7432cc640 1 -- 172.21.15.40:0/3778750001 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa73c0c3650 con 0x7fa73c0c24b0 2023-10-16T08:39:47.316 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.294+0000 7fa731ffb640 1 -- 172.21.15.40:0/3778750001 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa734015590 con 0x7fa73c0c24b0 2023-10-16T08:39:47.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.294+0000 7fa731ffb640 1 --2- 172.21.15.40:0/3778750001 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa7200664c0 0x7fa720068980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.294+0000 7fa731ffb640 1 -- 172.21.15.40:0/3778750001 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fa7340849d0 con 0x7fa73c0c24b0 2023-10-16T08:39:47.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.294+0000 7fa741068640 1 --2- 172.21.15.40:0/3778750001 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa7200664c0 0x7fa720068980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.317 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.294+0000 7fa741068640 1 --2- 172.21.15.40:0/3778750001 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa7200664c0 0x7fa720068980 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa738001130 tx=0x7fa738009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:47.318 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.298+0000 7fa731ffb640 1 -- 172.21.15.40:0/3778750001 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fa73c0c3650 con 0x7fa73c0c24b0 2023-10-16T08:39:47.457 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.450+0000 7fa7432cc640 1 -- 172.21.15.40:0/3778750001 --> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fa73c0c3960 con 0x7fa7200664c0 2023-10-16T08:39:47.458 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.450+0000 7fa731ffb640 1 -- 172.21.15.40:0/3778750001 <== mgr.4111 v2:172.21.15.86:6832/1188970899 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fa73c0c3960 con 0x7fa7200664c0 2023-10-16T08:39:47.458 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 5 to deep-scrub 2023-10-16T08:39:47.460 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.454+0000 7fa7432cc640 1 -- 172.21.15.40:0/3778750001 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa7200664c0 msgr2=0x7fa720068980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.454+0000 7fa7432cc640 1 --2- 172.21.15.40:0/3778750001 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa7200664c0 0x7fa720068980 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa738001130 tx=0x7fa738009040 comp rx=0 tx=0).stop 2023-10-16T08:39:47.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.454+0000 7fa7432cc640 1 -- 172.21.15.40:0/3778750001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa73c0c24b0 msgr2=0x7fa73c0c2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.461 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.454+0000 7fa7432cc640 1 --2- 172.21.15.40:0/3778750001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa73c0c24b0 0x7fa73c0c2960 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fa73c0b48b0 tx=0x7fa73400f580 comp rx=0 tx=0).stop 2023-10-16T08:39:47.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.454+0000 7fa7432cc640 1 -- 172.21.15.40:0/3778750001 shutdown_connections 2023-10-16T08:39:47.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.454+0000 7fa7432cc640 1 --2- 172.21.15.40:0/3778750001 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa7200664c0 0x7fa720068980 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.462 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.454+0000 7fa7432cc640 1 --2- 172.21.15.40:0/3778750001 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa73c0c24b0 0x7fa73c0c2960 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.454+0000 7fa7432cc640 1 --2- 172.21.15.40:0/3778750001 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa73c0c85a0 0x7fa73c0acba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.454+0000 7fa7432cc640 1 --2- 172.21.15.40:0/3778750001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa728003f50 0x7fa73c153350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.454+0000 7fa7432cc640 1 -- 172.21.15.40:0/3778750001 >> 172.21.15.40:0/3778750001 conn(0x7fa73c0cb400 msgr2=0x7fa73c15f850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:47.463 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.454+0000 7fa7432cc640 1 -- 172.21.15.40:0/3778750001 shutdown_connections 2023-10-16T08:39:47.464 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.454+0000 7fa7432cc640 1 -- 172.21.15.40:0/3778750001 wait complete. 2023-10-16T08:39:47.476 INFO:tasks.ceph:Scrubbing osd.6 2023-10-16T08:39:47.477 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-16T08:39:47.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.594+0000 7f834a31c640 1 Processor -- start 2023-10-16T08:39:47.601 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.594+0000 7f834a31c640 1 -- start start 2023-10-16T08:39:47.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.594+0000 7f834a31c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83440ce170 0x7f83441659b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.594+0000 7f834a31c640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f83440c1fc0 0x7f834406b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.594+0000 7f834a31c640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83440cf3d0 con 0x7f83440add60 2023-10-16T08:39:47.603 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.594+0000 7f834a31c640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8344165590 con 0x7f83440c1fc0 2023-10-16T08:39:47.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.594+0000 7f834a31c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8344164f60 con 0x7f83440ce170 2023-10-16T08:39:47.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.594+0000 7f83488b9640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f83440c1fc0 0x7f834406b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.604 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.594+0000 7f83437fe640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83440ce170 0x7f83441659b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.594+0000 7f83488b9640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f83440c1fc0 0x7f834406b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.40:59798/0 (socket says 172.21.15.40:59798) 2023-10-16T08:39:47.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.594+0000 7f8343fff640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f83440add60 0x7f83440cda60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.40:35862/0 (socket says 172.21.15.40:35862) 2023-10-16T08:39:47.605 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.594+0000 7f83488b9640 1 -- 172.21.15.40:0/761278836 learned_addr learned my addr 172.21.15.40:0/761278836 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:47.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.594+0000 7f83437fe640 1 -- 172.21.15.40:0/761278836 >> v2:172.21.15.86:3300/0 conn(0x7f83440c1fc0 msgr2=0x7f834406b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:47.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.594+0000 7f83437fe640 1 --2- 172.21.15.40:0/761278836 >> v2:172.21.15.86:3300/0 conn(0x7f83440c1fc0 0x7f834406b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.594+0000 7f83437fe640 1 -- 172.21.15.40:0/761278836 >> v1:172.21.15.86:6789/0 conn(0x7f83440add60 legacy=0x7f83440cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.606 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.594+0000 7f83437fe640 1 -- 172.21.15.40:0/761278836 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83440ae140 con 0x7f83440ce170 2023-10-16T08:39:47.607 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.594+0000 7f83488b9640 1 --2- 172.21.15.40:0/761278836 >> v2:172.21.15.86:3300/0 conn(0x7f83440c1fc0 0x7f834406b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:39:47.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.598+0000 7f83437fe640 1 --2- 172.21.15.40:0/761278836 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83440ce170 0x7f83441659b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f83340097c0 tx=0x7f833401c360 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7465b0aeaed05f3a server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:47.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.598+0000 7f8342ffd640 1 -- 172.21.15.40:0/761278836 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f834406c100 con 0x7f83440ce170 2023-10-16T08:39:47.610 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.598+0000 7f8342ffd640 1 -- 172.21.15.40:0/761278836 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83340a7050 con 0x7f83440ce170 2023-10-16T08:39:47.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.598+0000 7f8342ffd640 1 -- 172.21.15.40:0/761278836 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83340ab820 con 0x7f83440ce170 2023-10-16T08:39:47.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.598+0000 7f834a31c640 1 -- 172.21.15.40:0/761278836 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83440ce170 msgr2=0x7f83441659b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.611 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.598+0000 7f834a31c640 1 --2- 172.21.15.40:0/761278836 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83440ce170 0x7f83441659b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f83340097c0 tx=0x7f833401c360 comp rx=0 tx=0).stop 2023-10-16T08:39:47.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.598+0000 7f834a31c640 1 -- 172.21.15.40:0/761278836 shutdown_connections 2023-10-16T08:39:47.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.598+0000 7f834a31c640 1 --2- 172.21.15.40:0/761278836 >> v2:172.21.15.86:3300/0 conn(0x7f83440c1fc0 0x7f834406b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.612 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.598+0000 7f834a31c640 1 --2- 172.21.15.40:0/761278836 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83440ce170 0x7f83441659b0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.598+0000 7f834a31c640 1 -- 172.21.15.40:0/761278836 >> 172.21.15.40:0/761278836 conn(0x7f83440ce8b0 msgr2=0x7f83440cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:47.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.598+0000 7f834a31c640 1 -- 172.21.15.40:0/761278836 shutdown_connections 2023-10-16T08:39:47.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.598+0000 7f834a31c640 1 -- 172.21.15.40:0/761278836 wait complete. 2023-10-16T08:39:47.613 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f834a31c640 1 Processor -- start 2023-10-16T08:39:47.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f834a31c640 1 -- start start 2023-10-16T08:39:47.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f834a31c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83440add60 0x7f8344159770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.614 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f834a31c640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83440c1fc0 0x7f8344162480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f834a31c640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83440ce170 0x7f83441629c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f834a31c640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83440aeb30 con 0x7f83440ce170 2023-10-16T08:39:47.615 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f834a31c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8344159cb0 con 0x7f83440add60 2023-10-16T08:39:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f834a31c640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83440b4380 con 0x7f83440c1fc0 2023-10-16T08:39:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f83488b9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83440ce170 0x7f83441629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.616 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f83488b9640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83440ce170 0x7f83441629c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35748/0 (socket says 172.21.15.40:35748) 2023-10-16T08:39:47.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f83488b9640 1 -- 172.21.15.40:0/1336610855 learned_addr learned my addr 172.21.15.40:0/1336610855 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:47.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f83437fe640 1 --2- 172.21.15.40:0/1336610855 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83440c1fc0 0x7f8344162480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f8343fff640 1 --2- 172.21.15.40:0/1336610855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83440add60 0x7f8344159770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.617 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f83488b9640 1 -- 172.21.15.40:0/1336610855 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83440c1fc0 msgr2=0x7f8344162480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f83488b9640 1 --2- 172.21.15.40:0/1336610855 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f83440c1fc0 0x7f8344162480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f83488b9640 1 -- 172.21.15.40:0/1336610855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83440add60 msgr2=0x7f8344159770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.618 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f83488b9640 1 --2- 172.21.15.40:0/1336610855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f83440add60 0x7f8344159770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f83488b9640 1 -- 172.21.15.40:0/1336610855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833c009b30 con 0x7f83440ce170 2023-10-16T08:39:47.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f83488b9640 1 --2- 172.21.15.40:0/1336610855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83440ce170 0x7f83441629c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f833c00efc0 tx=0x7f833c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:47.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f8340ff9640 1 -- 172.21.15.40:0/1336610855 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c018070 con 0x7f83440ce170 2023-10-16T08:39:47.619 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f8340ff9640 1 -- 172.21.15.40:0/1336610855 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f833c00f040 con 0x7f83440ce170 2023-10-16T08:39:47.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f834a31c640 1 -- 172.21.15.40:0/1336610855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8334009420 con 0x7f83440ce170 2023-10-16T08:39:47.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f8340ff9640 1 -- 172.21.15.40:0/1336610855 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f833c00ce20 con 0x7f83440ce170 2023-10-16T08:39:47.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f834a31c640 1 -- 172.21.15.40:0/1336610855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83440ae4b0 con 0x7f83440ce170 2023-10-16T08:39:47.620 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f834a31c640 1 -- 172.21.15.40:0/1336610855 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f834416e2a0 con 0x7f83440ce170 2023-10-16T08:39:47.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.602+0000 7f8340ff9640 1 -- 172.21.15.40:0/1336610855 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f833c014480 con 0x7f83440ce170 2023-10-16T08:39:47.622 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.606+0000 7f8340ff9640 1 --2- 172.21.15.40:0/1336610855 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f83300664a0 0x7f8330068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.606+0000 7f8340ff9640 1 -- 172.21.15.40:0/1336610855 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f833c084da0 con 0x7f83440ce170 2023-10-16T08:39:47.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.606+0000 7f8340ff9640 1 --2- 172.21.15.40:0/1336610855 >> [v2:172.21.15.86:6813/2566952765,v1:172.21.15.86:6815/2566952765] conn(0x7f833006e820 0x7f8330070cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.606+0000 7f8340ff9640 1 -- 172.21.15.40:0/1336610855 --> [v2:172.21.15.86:6813/2566952765,v1:172.21.15.86:6815/2566952765] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8330071390 con 0x7f833006e820 2023-10-16T08:39:47.623 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.606+0000 7f8340ff9640 1 -- 172.21.15.40:0/1336610855 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f833c0851f0 con 0x7f83440ce170 2023-10-16T08:39:47.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.606+0000 7f8343fff640 1 --2- 172.21.15.40:0/1336610855 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f83300664a0 0x7f8330068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.606+0000 7f83437fe640 1 --2- 172.21.15.40:0/1336610855 >> [v2:172.21.15.86:6813/2566952765,v1:172.21.15.86:6815/2566952765] conn(0x7f833006e820 0x7f8330070cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.624 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.606+0000 7f8343fff640 1 --2- 172.21.15.40:0/1336610855 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f83300664a0 0x7f8330068960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8344163180 tx=0x7f8338007ca0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:47.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.606+0000 7f83437fe640 1 --2- 172.21.15.40:0/1336610855 >> [v2:172.21.15.86:6813/2566952765,v1:172.21.15.86:6815/2566952765] conn(0x7f833006e820 0x7f8330070cc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:47.625 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.606+0000 7f8340ff9640 1 -- 172.21.15.40:0/1336610855 <== osd.6 v2:172.21.15.86:6813/2566952765 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f8330071390 con 0x7f833006e820 2023-10-16T08:39:47.626 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.618+0000 7f834a31c640 1 -- 172.21.15.40:0/1336610855 --> [v2:172.21.15.86:6813/2566952765,v1:172.21.15.86:6815/2566952765] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f83440aec70 con 0x7f833006e820 2023-10-16T08:39:47.627 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.622+0000 7f8340ff9640 1 -- 172.21.15.40:0/1336610855 <== osd.6 v2:172.21.15.86:6813/2566952765 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f83440aec70 con 0x7f833006e820 2023-10-16T08:39:47.628 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-16T08:39:47.628 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T08:39:47.628 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-16T08:39:47.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.622+0000 7f834a31c640 1 -- 172.21.15.40:0/1336610855 >> [v2:172.21.15.86:6813/2566952765,v1:172.21.15.86:6815/2566952765] conn(0x7f833006e820 msgr2=0x7f8330070cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.629 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.622+0000 7f834a31c640 1 --2- 172.21.15.40:0/1336610855 >> [v2:172.21.15.86:6813/2566952765,v1:172.21.15.86:6815/2566952765] conn(0x7f833006e820 0x7f8330070cc0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.622+0000 7f834a31c640 1 -- 172.21.15.40:0/1336610855 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f83300664a0 msgr2=0x7f8330068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.622+0000 7f834a31c640 1 --2- 172.21.15.40:0/1336610855 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f83300664a0 0x7f8330068960 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8344163180 tx=0x7f8338007ca0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.622+0000 7f834a31c640 1 -- 172.21.15.40:0/1336610855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83440ce170 msgr2=0x7f83441629c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.630 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.622+0000 7f834a31c640 1 --2- 172.21.15.40:0/1336610855 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f83440ce170 0x7f83441629c0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f833c00efc0 tx=0x7f833c00c490 comp rx=0 tx=0).stop 2023-10-16T08:39:47.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.622+0000 7f83488b9640 1 -- 172.21.15.40:0/1336610855 reap_dead start 2023-10-16T08:39:47.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.622+0000 7f834a31c640 1 -- 172.21.15.40:0/1336610855 shutdown_connections 2023-10-16T08:39:47.631 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.622+0000 7f834a31c640 1 -- 172.21.15.40:0/1336610855 >> 172.21.15.40:0/1336610855 conn(0x7f83440ce8b0 msgr2=0x7f83440cec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:47.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.622+0000 7f834a31c640 1 -- 172.21.15.40:0/1336610855 shutdown_connections 2023-10-16T08:39:47.632 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.622+0000 7f834a31c640 1 -- 172.21.15.40:0/1336610855 wait complete. 2023-10-16T08:39:47.642 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-16T08:39:47.756 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.750+0000 7fbe4a2c0640 1 Processor -- start 2023-10-16T08:39:47.757 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.750+0000 7fbe4a2c0640 1 -- start start 2023-10-16T08:39:47.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.750+0000 7fbe4a2c0640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbe4415cdd0 0x7fbe440d0d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.750+0000 7fbe4a2c0640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbe440d12b0 0x7fbe440cda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.758 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.750+0000 7fbe4a2c0640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe440ce3f0 con 0x7fbe440cdf90 2023-10-16T08:39:47.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.750+0000 7fbe4a2c0640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe440d17c0 con 0x7fbe440d12b0 2023-10-16T08:39:47.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.750+0000 7fbe4a2c0640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe440d1aa0 con 0x7fbe4415cdd0 2023-10-16T08:39:47.759 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.750+0000 7fbe437fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbe440d12b0 0x7fbe440cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.750+0000 7fbe43fff640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fbe4415cdd0 0x7fbe440d0d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.750+0000 7fbe437fe640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7fbe440d12b0 0x7fbe440cda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35760/0 (socket says 172.21.15.40:35760) 2023-10-16T08:39:47.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.750+0000 7fbe437fe640 1 -- 172.21.15.40:0/936065524 learned_addr learned my addr 172.21.15.40:0/936065524 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:47.760 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.750+0000 7fbe437fe640 1 -- 172.21.15.40:0/936065524 >> v2:172.21.15.86:3300/0 conn(0x7fbe4415cdd0 msgr2=0x7fbe440d0d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:47.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.750+0000 7fbe437fe640 1 --2- 172.21.15.40:0/936065524 >> v2:172.21.15.86:3300/0 conn(0x7fbe4415cdd0 0x7fbe440d0d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.750+0000 7fbe437fe640 1 -- 172.21.15.40:0/936065524 >> v1:172.21.15.86:6789/0 conn(0x7fbe440cdf90 legacy=0x7fbe440b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.761 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.750+0000 7fbe437fe640 1 -- 172.21.15.40:0/936065524 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe440d1d80 con 0x7fbe440d12b0 2023-10-16T08:39:47.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.754+0000 7fbe437fe640 1 --2- 172.21.15.40:0/936065524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe440d12b0 0x7fbe440cda50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbe440d3870 tx=0x7fbe3401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2d9fd793fbc3cdd server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:47.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.754+0000 7fbe42ffd640 1 -- 172.21.15.40:0/936065524 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe340a5780 con 0x7fbe440d12b0 2023-10-16T08:39:47.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.754+0000 7fbe42ffd640 1 -- 172.21.15.40:0/936065524 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe340a58e0 con 0x7fbe440d12b0 2023-10-16T08:39:47.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.754+0000 7fbe42ffd640 1 -- 172.21.15.40:0/936065524 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe340a5c70 con 0x7fbe440d12b0 2023-10-16T08:39:47.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.754+0000 7fbe4a2c0640 1 -- 172.21.15.40:0/936065524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe440d12b0 msgr2=0x7fbe440cda50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.754+0000 7fbe4a2c0640 1 --2- 172.21.15.40:0/936065524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe440d12b0 0x7fbe440cda50 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fbe440d3870 tx=0x7fbe3401c9a0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.754+0000 7fbe4a2c0640 1 -- 172.21.15.40:0/936065524 shutdown_connections 2023-10-16T08:39:47.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.754+0000 7fbe4a2c0640 1 --2- 172.21.15.40:0/936065524 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe440d12b0 0x7fbe440cda50 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.754+0000 7fbe4a2c0640 1 --2- 172.21.15.40:0/936065524 >> v2:172.21.15.86:3300/0 conn(0x7fbe4415cdd0 0x7fbe440d0d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.754+0000 7fbe4a2c0640 1 -- 172.21.15.40:0/936065524 >> 172.21.15.40:0/936065524 conn(0x7fbe4406a290 msgr2=0x7fbe4406a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:47.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.754+0000 7fbe4a2c0640 1 -- 172.21.15.40:0/936065524 shutdown_connections 2023-10-16T08:39:47.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.754+0000 7fbe4a2c0640 1 -- 172.21.15.40:0/936065524 wait complete. 2023-10-16T08:39:47.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.754+0000 7fbe4a2c0640 1 Processor -- start 2023-10-16T08:39:47.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.754+0000 7fbe4a2c0640 1 -- start start 2023-10-16T08:39:47.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe4a2c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe440cdf90 0x7fbe4415b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe4a2c0640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe440d12b0 0x7fbe4413fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe4a2c0640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe4415cdd0 0x7fbe441548e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe4a2c0640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe440c75e0 con 0x7fbe4415cdd0 2023-10-16T08:39:47.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe4a2c0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe440c9240 con 0x7fbe440cdf90 2023-10-16T08:39:47.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe4a2c0640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe440c9520 con 0x7fbe440d12b0 2023-10-16T08:39:47.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe4885d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe4415cdd0 0x7fbe441548e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe4885d640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe4415cdd0 0x7fbe441548e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35774/0 (socket says 172.21.15.40:35774) 2023-10-16T08:39:47.771 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe4885d640 1 -- 172.21.15.40:0/744042881 learned_addr learned my addr 172.21.15.40:0/744042881 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:47.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe43fff640 1 --2- 172.21.15.40:0/744042881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe440cdf90 0x7fbe4415b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe437fe640 1 --2- 172.21.15.40:0/744042881 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe440d12b0 0x7fbe4413fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe4885d640 1 -- 172.21.15.40:0/744042881 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe440d12b0 msgr2=0x7fbe4413fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe4885d640 1 --2- 172.21.15.40:0/744042881 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe440d12b0 0x7fbe4413fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe4885d640 1 -- 172.21.15.40:0/744042881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe440cdf90 msgr2=0x7fbe4415b3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe4885d640 1 --2- 172.21.15.40:0/744042881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe440cdf90 0x7fbe4415b3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe4885d640 1 -- 172.21.15.40:0/744042881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe34009b30 con 0x7fbe4415cdd0 2023-10-16T08:39:47.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe4885d640 1 --2- 172.21.15.40:0/744042881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe4415cdd0 0x7fbe441548e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00c430 tx=0x7fbe3c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:47.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe40ff9640 1 -- 172.21.15.40:0/744042881 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c005220 con 0x7fbe4415cdd0 2023-10-16T08:39:47.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe40ff9640 1 -- 172.21.15.40:0/744042881 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe3c005380 con 0x7fbe4415cdd0 2023-10-16T08:39:47.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe4a2c0640 1 -- 172.21.15.40:0/744042881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe440c6f90 con 0x7fbe4415cdd0 2023-10-16T08:39:47.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe40ff9640 1 -- 172.21.15.40:0/744042881 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe3c007270 con 0x7fbe4415cdd0 2023-10-16T08:39:47.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe4a2c0640 1 -- 172.21.15.40:0/744042881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe44140290 con 0x7fbe4415cdd0 2023-10-16T08:39:47.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe4a2c0640 1 -- 172.21.15.40:0/744042881 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe440c7750 con 0x7fbe4415cdd0 2023-10-16T08:39:47.776 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe40ff9640 1 -- 172.21.15.40:0/744042881 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fbe3c0073d0 con 0x7fbe4415cdd0 2023-10-16T08:39:47.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe40ff9640 1 --2- 172.21.15.40:0/744042881 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbe240664a0 0x7fbe24068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:47.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe40ff9640 1 -- 172.21.15.40:0/744042881 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fbe3c0834f0 con 0x7fbe4415cdd0 2023-10-16T08:39:47.777 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.758+0000 7fbe43fff640 1 --2- 172.21.15.40:0/744042881 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbe240664a0 0x7fbe24068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:47.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.762+0000 7fbe43fff640 1 --2- 172.21.15.40:0/744042881 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbe240664a0 0x7fbe24068960 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009a50 tx=0x7fbe2c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:47.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.762+0000 7fbe40ff9640 1 -- 172.21.15.40:0/744042881 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fbe3c0508e0 con 0x7fbe4415cdd0 2023-10-16T08:39:47.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.914+0000 7fbe4a2c0640 1 -- 172.21.15.40:0/744042881 --> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fbe441404e0 con 0x7fbe240664a0 2023-10-16T08:39:47.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.914+0000 7fbe40ff9640 1 -- 172.21.15.40:0/744042881 <== mgr.4111 v2:172.21.15.86:6832/1188970899 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fbe441404e0 con 0x7fbe240664a0 2023-10-16T08:39:47.923 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 6 to deep-scrub 2023-10-16T08:39:47.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.918+0000 7fbe4a2c0640 1 -- 172.21.15.40:0/744042881 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbe240664a0 msgr2=0x7fbe24068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.924 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.918+0000 7fbe4a2c0640 1 --2- 172.21.15.40:0/744042881 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbe240664a0 0x7fbe24068960 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fbe2c009a50 tx=0x7fbe2c008040 comp rx=0 tx=0).stop 2023-10-16T08:39:47.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.918+0000 7fbe4a2c0640 1 -- 172.21.15.40:0/744042881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe4415cdd0 msgr2=0x7fbe441548e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:47.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.918+0000 7fbe4a2c0640 1 --2- 172.21.15.40:0/744042881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe4415cdd0 0x7fbe441548e0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fbe3c00c430 tx=0x7fbe3c00c8f0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.925 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.918+0000 7fbe4a2c0640 1 -- 172.21.15.40:0/744042881 shutdown_connections 2023-10-16T08:39:47.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.918+0000 7fbe4a2c0640 1 --2- 172.21.15.40:0/744042881 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fbe240664a0 0x7fbe24068960 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.926 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.918+0000 7fbe4a2c0640 1 --2- 172.21.15.40:0/744042881 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fbe4415cdd0 0x7fbe441548e0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.918+0000 7fbe4a2c0640 1 --2- 172.21.15.40:0/744042881 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fbe440d12b0 0x7fbe4413fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.918+0000 7fbe4a2c0640 1 --2- 172.21.15.40:0/744042881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbe440cdf90 0x7fbe4415b3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:47.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.918+0000 7fbe4a2c0640 1 -- 172.21.15.40:0/744042881 >> 172.21.15.40:0/744042881 conn(0x7fbe4406a290 msgr2=0x7fbe44150ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:47.927 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.918+0000 7fbe4a2c0640 1 -- 172.21.15.40:0/744042881 shutdown_connections 2023-10-16T08:39:47.928 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:47.918+0000 7fbe4a2c0640 1 -- 172.21.15.40:0/744042881 wait complete. 2023-10-16T08:39:47.940 INFO:tasks.ceph:Scrubbing osd.7 2023-10-16T08:39:47.941 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-16T08:39:48.057 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.050+0000 7f655b1ef640 1 Processor -- start 2023-10-16T08:39:48.058 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.050+0000 7f655b1ef640 1 -- start start 2023-10-16T08:39:48.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.050+0000 7f655b1ef640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f65540acff0 0x7f65540add00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:48.059 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.050+0000 7f655b1ef640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655406bb80 con 0x7f655415ce20 2023-10-16T08:39:48.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.050+0000 7f655b1ef640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f655406fa60 con 0x7f65540cf5d0 2023-10-16T08:39:48.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.050+0000 7f655b1ef640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f655406cf80 con 0x7f65540acff0 2023-10-16T08:39:48.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.050+0000 7f655978c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f65540acff0 0x7f65540add00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:48.060 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.050+0000 7f6558f8b640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f655415ce20 0x7f655406f350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34088/0 (socket says 172.21.15.40:34088) 2023-10-16T08:39:48.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.050+0000 7f655978c640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f65540acff0 0x7f65540add00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35780/0 (socket says 172.21.15.40:35780) 2023-10-16T08:39:48.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.050+0000 7f6558f8b640 1 -- 172.21.15.40:0/1555526846 learned_addr learned my addr 172.21.15.40:0/1555526846 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:48.061 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.054+0000 7f655978c640 1 -- 172.21.15.40:0/1555526846 >> v1:172.21.15.86:6789/0 conn(0x7f65540cf5d0 legacy=0x7f65540ac8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:48.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.054+0000 7f655978c640 1 -- 172.21.15.40:0/1555526846 >> v1:172.21.15.40:6789/0 conn(0x7f655415ce20 legacy=0x7f655406f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:48.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.054+0000 7f655978c640 1 -- 172.21.15.40:0/1555526846 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655406b530 con 0x7f65540acff0 2023-10-16T08:39:48.062 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.054+0000 7f65537fe640 1 -- 172.21.15.40:0/1555526846 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2462952176 0 0) 0x7f655406bb80 con 0x7f655415ce20 2023-10-16T08:39:48.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.054+0000 7f655978c640 1 --2- 172.21.15.40:0/1555526846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65540acff0 0x7f65540add00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f654c009d90 tx=0x7f654c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e44052bee505999 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:48.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.054+0000 7f65537fe640 1 -- 172.21.15.40:0/1555526846 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c0a59c0 con 0x7f65540acff0 2023-10-16T08:39:48.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.054+0000 7f65537fe640 1 -- 172.21.15.40:0/1555526846 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f654c0a5b20 con 0x7f65540acff0 2023-10-16T08:39:48.064 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.054+0000 7f65537fe640 1 -- 172.21.15.40:0/1555526846 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f654c0ad440 con 0x7f65540acff0 2023-10-16T08:39:48.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.054+0000 7f655b1ef640 1 -- 172.21.15.40:0/1555526846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65540acff0 msgr2=0x7f65540add00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:48.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.054+0000 7f655b1ef640 1 --2- 172.21.15.40:0/1555526846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65540acff0 0x7f65540add00 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f654c009d90 tx=0x7f654c01c8e0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.054+0000 7f655b1ef640 1 -- 172.21.15.40:0/1555526846 shutdown_connections 2023-10-16T08:39:48.065 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.054+0000 7f655b1ef640 1 --2- 172.21.15.40:0/1555526846 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f65540acff0 0x7f65540add00 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.054+0000 7f655b1ef640 1 -- 172.21.15.40:0/1555526846 >> 172.21.15.40:0/1555526846 conn(0x7f65540ce8b0 msgr2=0x7f65540cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:48.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.054+0000 7f655b1ef640 1 -- 172.21.15.40:0/1555526846 shutdown_connections 2023-10-16T08:39:48.066 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.054+0000 7f655b1ef640 1 -- 172.21.15.40:0/1555526846 wait complete. 2023-10-16T08:39:48.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f655b1ef640 1 Processor -- start 2023-10-16T08:39:48.067 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f655b1ef640 1 -- start start 2023-10-16T08:39:48.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f655b1ef640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65540acff0 0x7f655415f9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:48.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f655b1ef640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655415ce20 0x7f6554149f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:48.070 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f655b1ef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f655414ccb0 0x7f65540b27f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:48.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f655b1ef640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655414a8e0 con 0x7f655415ce20 2023-10-16T08:39:48.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f655b1ef640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f655415eb20 con 0x7f655414ccb0 2023-10-16T08:39:48.071 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f655b1ef640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f655415ee00 con 0x7f65540acff0 2023-10-16T08:39:48.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f6553fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655415ce20 0x7f6554149f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:48.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f6553fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655415ce20 0x7f6554149f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35796/0 (socket says 172.21.15.40:35796) 2023-10-16T08:39:48.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f6553fff640 1 -- 172.21.15.40:0/212172703 learned_addr learned my addr 172.21.15.40:0/212172703 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:48.072 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f655978c640 1 --2- 172.21.15.40:0/212172703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f655414ccb0 0x7f65540b27f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:48.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f6558f8b640 1 --2- 172.21.15.40:0/212172703 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65540acff0 0x7f655415f9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:48.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f6553fff640 1 -- 172.21.15.40:0/212172703 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65540acff0 msgr2=0x7f655415f9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:48.073 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f6553fff640 1 --2- 172.21.15.40:0/212172703 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f65540acff0 0x7f655415f9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f6553fff640 1 -- 172.21.15.40:0/212172703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f655414ccb0 msgr2=0x7f65540b27f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:48.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f6553fff640 1 --2- 172.21.15.40:0/212172703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f655414ccb0 0x7f65540b27f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f6553fff640 1 -- 172.21.15.40:0/212172703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f654c009a70 con 0x7f655415ce20 2023-10-16T08:39:48.074 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f655978c640 1 --2- 172.21.15.40:0/212172703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f655414ccb0 0x7f65540b27f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:39:48.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f6553fff640 1 --2- 172.21.15.40:0/212172703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655415ce20 0x7f6554149f10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f654000c450 tx=0x7f654000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:48.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f65517fa640 1 -- 172.21.15.40:0/212172703 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6540012830 con 0x7f655415ce20 2023-10-16T08:39:48.075 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f65517fa640 1 -- 172.21.15.40:0/212172703 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65400129b0 con 0x7f655415ce20 2023-10-16T08:39:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f655b1ef640 1 -- 172.21.15.40:0/212172703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f655415f0e0 con 0x7f655415ce20 2023-10-16T08:39:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f65517fa640 1 -- 172.21.15.40:0/212172703 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6540012d60 con 0x7f655415ce20 2023-10-16T08:39:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f655b1ef640 1 -- 172.21.15.40:0/212172703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65540b2d30 con 0x7f655415ce20 2023-10-16T08:39:48.076 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.058+0000 7f655b1ef640 1 -- 172.21.15.40:0/212172703 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f655415f320 con 0x7f655415ce20 2023-10-16T08:39:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.062+0000 7f65517fa640 1 -- 172.21.15.40:0/212172703 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f654001c460 con 0x7f655415ce20 2023-10-16T08:39:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.062+0000 7f65517fa640 1 --2- 172.21.15.40:0/212172703 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f65380664d0 0x7f6538068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.062+0000 7f65517fa640 1 -- 172.21.15.40:0/212172703 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f65400847d0 con 0x7f655415ce20 2023-10-16T08:39:48.077 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.062+0000 7f65517fa640 1 --2- 172.21.15.40:0/212172703 >> [v2:172.21.15.86:6800/2560845614,v1:172.21.15.86:6801/2560845614] conn(0x7f653806e850 0x7f6538070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:48.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.062+0000 7f65517fa640 1 -- 172.21.15.40:0/212172703 --> [v2:172.21.15.86:6800/2560845614,v1:172.21.15.86:6801/2560845614] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f65380713c0 con 0x7f653806e850 2023-10-16T08:39:48.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.062+0000 7f65517fa640 1 -- 172.21.15.40:0/212172703 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_get_version_reply(handle=1 version=22) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6540084c20 con 0x7f655415ce20 2023-10-16T08:39:48.078 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.062+0000 7f6558f8b640 1 --2- 172.21.15.40:0/212172703 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f65380664d0 0x7f6538068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.062+0000 7f655978c640 1 --2- 172.21.15.40:0/212172703 >> [v2:172.21.15.86:6800/2560845614,v1:172.21.15.86:6801/2560845614] conn(0x7f653806e850 0x7f6538070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.062+0000 7f6558f8b640 1 --2- 172.21.15.40:0/212172703 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f65380664d0 0x7f6538068990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6548007fd0 tx=0x7f6548009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.062+0000 7f655978c640 1 --2- 172.21.15.40:0/212172703 >> [v2:172.21.15.86:6800/2560845614,v1:172.21.15.86:6801/2560845614] conn(0x7f653806e850 0x7f6538070cf0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:48.079 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.062+0000 7f65517fa640 1 -- 172.21.15.40:0/212172703 <== osd.7 v2:172.21.15.86:6800/2560845614 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25607 (crc 0 0 0) 0x7f65380713c0 con 0x7f653806e850 2023-10-16T08:39:48.082 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.074+0000 7f655b1ef640 1 -- 172.21.15.40:0/212172703 --> [v2:172.21.15.86:6800/2560845614,v1:172.21.15.86:6801/2560845614] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f655404e9c0 con 0x7f653806e850 2023-10-16T08:39:48.083 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.078+0000 7f65517fa640 1 -- 172.21.15.40:0/212172703 <== osd.7 v2:172.21.15.86:6800/2560845614 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f655404e9c0 con 0x7f653806e850 2023-10-16T08:39:48.084 INFO:teuthology.orchestra.run.smithi040.stdout:{ 2023-10-16T08:39:48.084 INFO:teuthology.orchestra.run.smithi040.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-16T08:39:48.084 INFO:teuthology.orchestra.run.smithi040.stdout:} 2023-10-16T08:39:48.085 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.078+0000 7f655b1ef640 1 -- 172.21.15.40:0/212172703 >> [v2:172.21.15.86:6800/2560845614,v1:172.21.15.86:6801/2560845614] conn(0x7f653806e850 msgr2=0x7f6538070cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:48.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.078+0000 7f655b1ef640 1 --2- 172.21.15.40:0/212172703 >> [v2:172.21.15.86:6800/2560845614,v1:172.21.15.86:6801/2560845614] conn(0x7f653806e850 0x7f6538070cf0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.078+0000 7f655b1ef640 1 -- 172.21.15.40:0/212172703 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f65380664d0 msgr2=0x7f6538068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:48.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.078+0000 7f655b1ef640 1 --2- 172.21.15.40:0/212172703 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f65380664d0 0x7f6538068990 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6548007fd0 tx=0x7f6548009040 comp rx=0 tx=0).stop 2023-10-16T08:39:48.086 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.078+0000 7f655b1ef640 1 -- 172.21.15.40:0/212172703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655415ce20 msgr2=0x7f6554149f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:48.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.078+0000 7f655b1ef640 1 --2- 172.21.15.40:0/212172703 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f655415ce20 0x7f6554149f10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f654000c450 tx=0x7f654000c910 comp rx=0 tx=0).stop 2023-10-16T08:39:48.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.078+0000 7f655978c640 1 -- 172.21.15.40:0/212172703 reap_dead start 2023-10-16T08:39:48.087 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.078+0000 7f655b1ef640 1 -- 172.21.15.40:0/212172703 shutdown_connections 2023-10-16T08:39:48.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.078+0000 7f655b1ef640 1 -- 172.21.15.40:0/212172703 >> 172.21.15.40:0/212172703 conn(0x7f65540ce8b0 msgr2=0x7f6554157ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:48.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.078+0000 7f655b1ef640 1 -- 172.21.15.40:0/212172703 shutdown_connections 2023-10-16T08:39:48.088 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.078+0000 7f655b1ef640 1 -- 172.21.15.40:0/212172703 wait complete. 2023-10-16T08:39:48.098 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-16T08:39:48.209 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.202+0000 7fe7b1b17640 1 Processor -- start 2023-10-16T08:39:48.210 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.202+0000 7fe7b1b17640 1 -- start start 2023-10-16T08:39:48.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.202+0000 7fe7b1b17640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7ac0cddc0 0x7fe7ac0d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:48.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.202+0000 7fe7b1b17640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ac0b7f80 con 0x7fe7ac15cce0 2023-10-16T08:39:48.212 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.202+0000 7fe7b1b17640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ac0b7370 con 0x7fe7ac0cd960 2023-10-16T08:39:48.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.202+0000 7fe7b1b17640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac0b7650 con 0x7fe7ac0cddc0 2023-10-16T08:39:48.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.202+0000 7fe7ab7fe640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fe7ac15cce0 0x7fe7ac0b68d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:34092/0 (socket says 172.21.15.40:34092) 2023-10-16T08:39:48.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.202+0000 7fe7ab7fe640 1 -- 172.21.15.40:0/2843274637 learned_addr learned my addr 172.21.15.40:0/2843274637 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:39:48.213 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7abfff640 1 --2- 172.21.15.40:0/2843274637 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7ac0cddc0 0x7fe7ac0d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7aaffd640 1 -- 172.21.15.40:0/2843274637 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 476820647 0 0) 0x7fe7ac0b7f80 con 0x7fe7ac15cce0 2023-10-16T08:39:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7aaffd640 1 -- 172.21.15.40:0/2843274637 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe798003530 con 0x7fe7ac15cce0 2023-10-16T08:39:48.214 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7aaffd640 1 -- 172.21.15.40:0/2843274637 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1876993198 0 0) 0x7fe798003530 con 0x7fe7ac15cce0 2023-10-16T08:39:48.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7aaffd640 1 -- 172.21.15.40:0/2843274637 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7ac0cddc0 msgr2=0x7fe7ac0d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:48.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7aaffd640 1 --2- 172.21.15.40:0/2843274637 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7ac0cddc0 0x7fe7ac0d1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7aaffd640 1 -- 172.21.15.40:0/2843274637 >> v1:172.21.15.86:6789/0 conn(0x7fe7ac0cd960 legacy=0x7fe7ac0d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:48.215 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7aaffd640 1 -- 172.21.15.40:0/2843274637 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ac0b7930 con 0x7fe7ac15cce0 2023-10-16T08:39:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7aaffd640 1 -- 172.21.15.40:0/2843274637 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7fe79c001680 con 0x7fe7ac15cce0 2023-10-16T08:39:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7aaffd640 1 -- 172.21.15.40:0/2843274637 >> v1:172.21.15.40:6789/0 conn(0x7fe7ac15cce0 legacy=0x7fe7ac0b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:48.216 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7aaffd640 1 --2- 172.21.15.40:0/2843274637 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7ac0cddc0 0x7fe798003a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7aaffd640 1 --2- 172.21.15.40:0/2843274637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798004050 0x7fe7980047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7aaffd640 1 --2- 172.21.15.40:0/2843274637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe798004d60 0x7fe7980051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7aaffd640 1 -- 172.21.15.40:0/2843274637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac0b7650 con 0x7fe798004050 2023-10-16T08:39:48.217 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7aaffd640 1 -- 172.21.15.40:0/2843274637 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe79800bba0 con 0x7fe798004d60 2023-10-16T08:39:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7aaffd640 1 -- 172.21.15.40:0/2843274637 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe79800bd60 con 0x7fe7ac0cddc0 2023-10-16T08:39:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7a3fff640 1 --2- 172.21.15.40:0/2843274637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798004050 0x7fe7980047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7abfff640 1 --2- 172.21.15.40:0/2843274637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe798004d60 0x7fe7980051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:48.218 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7ab7fe640 1 --2- 172.21.15.40:0/2843274637 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7ac0cddc0 0x7fe798003a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:48.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7a3fff640 1 -- 172.21.15.40:0/2843274637 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7ac0cddc0 msgr2=0x7fe798003a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:48.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7a3fff640 1 --2- 172.21.15.40:0/2843274637 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7ac0cddc0 0x7fe798003a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.219 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7a3fff640 1 -- 172.21.15.40:0/2843274637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe798004d60 msgr2=0x7fe7980051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:48.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7a3fff640 1 --2- 172.21.15.40:0/2843274637 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe798004d60 0x7fe7980051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.220 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7a3fff640 1 -- 172.21.15.40:0/2843274637 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe798003530 con 0x7fe798004050 2023-10-16T08:39:48.223 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7a3fff640 1 --2- 172.21.15.40:0/2843274637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798004050 0x7fe7980047d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe794001780 tx=0x7fe79401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d8356111d02ea05 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:48.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7aaffd640 1 -- 172.21.15.40:0/2843274637 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7940052d0 con 0x7fe798004050 2023-10-16T08:39:48.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7aaffd640 1 -- 172.21.15.40:0/2843274637 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7940a7050 con 0x7fe798004050 2023-10-16T08:39:48.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7aaffd640 1 -- 172.21.15.40:0/2843274637 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe794003600 con 0x7fe798004050 2023-10-16T08:39:48.224 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7b1b17640 1 -- 172.21.15.40:0/2843274637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798004050 msgr2=0x7fe7980047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:48.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7b1b17640 1 --2- 172.21.15.40:0/2843274637 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798004050 0x7fe7980047d0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fe794001780 tx=0x7fe79401c400 comp rx=0 tx=0).stop 2023-10-16T08:39:48.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7abfff640 1 -- 172.21.15.40:0/2843274637 reap_dead start 2023-10-16T08:39:48.225 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7b1b17640 1 -- 172.21.15.40:0/2843274637 shutdown_connections 2023-10-16T08:39:48.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7b1b17640 1 -- 172.21.15.40:0/2843274637 >> 172.21.15.40:0/2843274637 conn(0x7fe7ac06a290 msgr2=0x7fe7ac06a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:48.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7b1b17640 1 -- 172.21.15.40:0/2843274637 shutdown_connections 2023-10-16T08:39:48.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.206+0000 7fe7b1b17640 1 -- 172.21.15.40:0/2843274637 wait complete. 2023-10-16T08:39:48.226 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7b1b17640 1 Processor -- start 2023-10-16T08:39:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7b1b17640 1 -- start start 2023-10-16T08:39:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7b1b17640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798004050 0x7fe7ac158180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7b1b17640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7ac145090 0x7fe7ac145510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:48.227 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7b1b17640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7ac1f2790 0x7fe7ac1f2c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:48.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7b1b17640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac1f3500 con 0x7fe798004050 2023-10-16T08:39:48.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7b1b17640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac145a50 con 0x7fe7ac145090 2023-10-16T08:39:48.228 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7b1b17640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac142b50 con 0x7fe7ac1f2790 2023-10-16T08:39:48.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7ab7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798004050 0x7fe7ac158180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:48.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7ab7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798004050 0x7fe7ac158180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35820/0 (socket says 172.21.15.40:35820) 2023-10-16T08:39:48.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7ab7fe640 1 -- 172.21.15.40:0/3437856389 learned_addr learned my addr 172.21.15.40:0/3437856389 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:48.229 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7a3fff640 1 --2- 172.21.15.40:0/3437856389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7ac145090 0x7fe7ac145510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:48.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7abfff640 1 --2- 172.21.15.40:0/3437856389 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7ac1f2790 0x7fe7ac1f2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:48.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7ab7fe640 1 -- 172.21.15.40:0/3437856389 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7ac1f2790 msgr2=0x7fe7ac1f2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:48.230 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7ab7fe640 1 --2- 172.21.15.40:0/3437856389 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7ac1f2790 0x7fe7ac1f2c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7ab7fe640 1 -- 172.21.15.40:0/3437856389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7ac145090 msgr2=0x7fe7ac145510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7ab7fe640 1 --2- 172.21.15.40:0/3437856389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7ac145090 0x7fe7ac145510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7ab7fe640 1 -- 172.21.15.40:0/3437856389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ac141d90 con 0x7fe798004050 2023-10-16T08:39:48.231 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7ab7fe640 1 --2- 172.21.15.40:0/3437856389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798004050 0x7fe7ac158180 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0117f0 tx=0x7fe79c011cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:48.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7a8ff9640 1 -- 172.21.15.40:0/3437856389 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c00fb60 con 0x7fe798004050 2023-10-16T08:39:48.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7a8ff9640 1 -- 172.21.15.40:0/3437856389 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79c00fce0 con 0x7fe798004050 2023-10-16T08:39:48.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7b1b17640 1 -- 172.21.15.40:0/3437856389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ac142070 con 0x7fe798004050 2023-10-16T08:39:48.232 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7a8ff9640 1 -- 172.21.15.40:0/3437856389 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c015650 con 0x7fe798004050 2023-10-16T08:39:48.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7b1b17640 1 -- 172.21.15.40:0/3437856389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ac142630 con 0x7fe798004050 2023-10-16T08:39:48.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7b1b17640 1 -- 172.21.15.40:0/3437856389 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7ac04e9c0 con 0x7fe798004050 2023-10-16T08:39:48.233 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.210+0000 7fe7a8ff9640 1 -- 172.21.15.40:0/3437856389 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fe79c0157b0 con 0x7fe798004050 2023-10-16T08:39:48.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.214+0000 7fe7a8ff9640 1 --2- 172.21.15.40:0/3437856389 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe78c0664a0 0x7fe78c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:48.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.214+0000 7fe7a8ff9640 1 -- 172.21.15.40:0/3437856389 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fe79c087280 con 0x7fe798004050 2023-10-16T08:39:48.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.214+0000 7fe7a3fff640 1 --2- 172.21.15.40:0/3437856389 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe78c0664a0 0x7fe78c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:48.234 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.214+0000 7fe7a3fff640 1 --2- 172.21.15.40:0/3437856389 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe78c0664a0 0x7fe78c068960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe794001750 tx=0x7fe7940011a0 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:48.235 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.218+0000 7fe7a8ff9640 1 -- 172.21.15.40:0/3437856389 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fe79c054670 con 0x7fe798004050 2023-10-16T08:39:48.375 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.370+0000 7fe7b1b17640 1 -- 172.21.15.40:0/3437856389 --> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7ac0ce250 con 0x7fe78c0664a0 2023-10-16T08:39:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.370+0000 7fe7a8ff9640 1 -- 172.21.15.40:0/3437856389 <== mgr.4111 v2:172.21.15.86:6832/1188970899 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fe794005270 con 0x7fe78c0664a0 2023-10-16T08:39:48.376 INFO:teuthology.orchestra.run.smithi040.stderr:instructed osd(s) 7 to deep-scrub 2023-10-16T08:39:48.378 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.370+0000 7fe7b1b17640 1 -- 172.21.15.40:0/3437856389 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe78c0664a0 msgr2=0x7fe78c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:48.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.370+0000 7fe7b1b17640 1 --2- 172.21.15.40:0/3437856389 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe78c0664a0 0x7fe78c068960 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe794001750 tx=0x7fe7940011a0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.370+0000 7fe7b1b17640 1 -- 172.21.15.40:0/3437856389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798004050 msgr2=0x7fe7ac158180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:48.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.370+0000 7fe7b1b17640 1 --2- 172.21.15.40:0/3437856389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798004050 0x7fe7ac158180 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe79c0117f0 tx=0x7fe79c011cb0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.379 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.370+0000 7fe7b1b17640 1 -- 172.21.15.40:0/3437856389 shutdown_connections 2023-10-16T08:39:48.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.370+0000 7fe7b1b17640 1 --2- 172.21.15.40:0/3437856389 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fe7ac1f2790 0x7fe7ac1f2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.370+0000 7fe7b1b17640 1 --2- 172.21.15.40:0/3437856389 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe7ac145090 0x7fe7ac145510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.370+0000 7fe7b1b17640 1 --2- 172.21.15.40:0/3437856389 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fe78c0664a0 0x7fe78c068960 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.380 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.370+0000 7fe7b1b17640 1 --2- 172.21.15.40:0/3437856389 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fe798004050 0x7fe7ac158180 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.370+0000 7fe7b1b17640 1 -- 172.21.15.40:0/3437856389 >> 172.21.15.40:0/3437856389 conn(0x7fe7ac06a290 msgr2=0x7fe7ac0d1650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:48.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.370+0000 7fe7b1b17640 1 -- 172.21.15.40:0/3437856389 shutdown_connections 2023-10-16T08:39:48.381 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.370+0000 7fe7b1b17640 1 -- 172.21.15.40:0/3437856389 wait complete. 2023-10-16T08:39:48.393 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:39:48.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.502+0000 7f25d8617640 1 Processor -- start 2023-10-16T08:39:48.511 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.502+0000 7f25d8617640 1 -- start start 2023-10-16T08:39:48.512 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.506+0000 7f25d8617640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f25d015d180 0x7f25d006b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:48.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.506+0000 7f25d8617640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f25d015d560 0x7f25d006fb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:48.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.506+0000 7f25d8617640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25d0070090 con 0x7f25d00ad820 2023-10-16T08:39:48.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.506+0000 7f25d8617640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d00cf480 con 0x7f25d015d560 2023-10-16T08:39:48.513 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.506+0000 7f25d8617640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f25d00cf760 con 0x7f25d015d180 2023-10-16T08:39:48.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.506+0000 7f25d5bb2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f25d015d560 0x7f25d006fb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:48.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.506+0000 7f25d5bb2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f25d015d560 0x7f25d006fb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35836/0 (socket says 172.21.15.40:35836) 2023-10-16T08:39:48.514 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.506+0000 7f25d5bb2640 1 -- 172.21.15.40:0/3319500367 learned_addr learned my addr 172.21.15.40:0/3319500367 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:48.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.506+0000 7f25d63b3640 1 --2- 172.21.15.40:0/3319500367 >> v2:172.21.15.86:3300/0 conn(0x7f25d015d180 0x7f25d006b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:48.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.506+0000 7f25d5bb2640 1 -- 172.21.15.40:0/3319500367 >> v2:172.21.15.86:3300/0 conn(0x7f25d015d180 msgr2=0x7f25d006b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:48.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.506+0000 7f25d5bb2640 1 --2- 172.21.15.40:0/3319500367 >> v2:172.21.15.86:3300/0 conn(0x7f25d015d180 0x7f25d006b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.515 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.506+0000 7f25d5bb2640 1 -- 172.21.15.40:0/3319500367 >> v1:172.21.15.86:6789/0 conn(0x7f25d00ad820 legacy=0x7f25d00c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:39:48.516 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.506+0000 7f25d5bb2640 1 -- 172.21.15.40:0/3319500367 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25d00c8fb0 con 0x7f25d015d560 2023-10-16T08:39:48.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.506+0000 7f25d5bb2640 1 --2- 172.21.15.40:0/3319500367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d015d560 0x7f25d006fb50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f25d006a0d0 tx=0x7f25cc01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80034253abe4357d server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:48.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.506+0000 7f25d53b1640 1 -- 172.21.15.40:0/3319500367 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc0ab670 con 0x7f25d015d560 2023-10-16T08:39:48.519 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.506+0000 7f25d53b1640 1 -- 172.21.15.40:0/3319500367 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25cc0ab7d0 con 0x7f25d015d560 2023-10-16T08:39:48.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.506+0000 7f25d53b1640 1 -- 172.21.15.40:0/3319500367 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc0abb60 con 0x7f25d015d560 2023-10-16T08:39:48.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d8617640 1 -- 172.21.15.40:0/3319500367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d015d560 msgr2=0x7f25d006fb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:48.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d8617640 1 --2- 172.21.15.40:0/3319500367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d015d560 0x7f25d006fb50 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f25d006a0d0 tx=0x7f25cc01c490 comp rx=0 tx=0).stop 2023-10-16T08:39:48.520 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d8617640 1 -- 172.21.15.40:0/3319500367 shutdown_connections 2023-10-16T08:39:48.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d8617640 1 --2- 172.21.15.40:0/3319500367 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d015d560 0x7f25d006fb50 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d8617640 1 --2- 172.21.15.40:0/3319500367 >> v2:172.21.15.86:3300/0 conn(0x7f25d015d180 0x7f25d006b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.521 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d8617640 1 -- 172.21.15.40:0/3319500367 >> 172.21.15.40:0/3319500367 conn(0x7f25d00ce960 msgr2=0x7f25d00ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:48.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d8617640 1 -- 172.21.15.40:0/3319500367 shutdown_connections 2023-10-16T08:39:48.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d8617640 1 -- 172.21.15.40:0/3319500367 wait complete. 2023-10-16T08:39:48.522 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d8617640 1 Processor -- start 2023-10-16T08:39:48.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d8617640 1 -- start start 2023-10-16T08:39:48.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d8617640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25d00ad820 0x7f25d00c4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:48.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d8617640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d015d180 0x7f25d0157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:48.523 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d8617640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f25d015d560 0x7f25d0158190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:48.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d8617640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d00b4340 con 0x7f25d015d180 2023-10-16T08:39:48.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d8617640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25d00b1810 con 0x7f25d00ad820 2023-10-16T08:39:48.524 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d8617640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25d00b39a0 con 0x7f25d015d560 2023-10-16T08:39:48.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d5bb2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d015d180 0x7f25d0157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:48.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d5bb2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d015d180 0x7f25d0157c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:35844/0 (socket says 172.21.15.40:35844) 2023-10-16T08:39:48.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d5bb2640 1 -- 172.21.15.40:0/3769030368 learned_addr learned my addr 172.21.15.40:0/3769030368 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:39:48.525 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d63b3640 1 --2- 172.21.15.40:0/3769030368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25d00ad820 0x7f25d00c4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:48.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d6bb4640 1 --2- 172.21.15.40:0/3769030368 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f25d015d560 0x7f25d0158190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:48.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d5bb2640 1 -- 172.21.15.40:0/3769030368 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f25d015d560 msgr2=0x7f25d0158190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:48.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d5bb2640 1 --2- 172.21.15.40:0/3769030368 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f25d015d560 0x7f25d0158190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.526 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d5bb2640 1 -- 172.21.15.40:0/3769030368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25d00ad820 msgr2=0x7f25d00c4cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d5bb2640 1 --2- 172.21.15.40:0/3769030368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25d00ad820 0x7f25d00c4cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d5bb2640 1 -- 172.21.15.40:0/3769030368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25cc0094d0 con 0x7f25d015d180 2023-10-16T08:39:48.527 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d5bb2640 1 --2- 172.21.15.40:0/3769030368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d015d180 0x7f25d0157c50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0ad040 tx=0x7f25cc01acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25c6ffd640 1 -- 172.21.15.40:0/3769030368 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc0b4800 con 0x7f25d015d180 2023-10-16T08:39:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25c6ffd640 1 -- 172.21.15.40:0/3769030368 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25cc0b4980 con 0x7f25d015d180 2023-10-16T08:39:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d8617640 1 -- 172.21.15.40:0/3769030368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25d00b3c80 con 0x7f25d015d180 2023-10-16T08:39:48.528 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25c6ffd640 1 -- 172.21.15.40:0/3769030368 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25cc0b4d30 con 0x7f25d015d180 2023-10-16T08:39:48.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.510+0000 7f25d8617640 1 -- 172.21.15.40:0/3769030368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25d00c1e70 con 0x7f25d015d180 2023-10-16T08:39:48.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.514+0000 7f25d8617640 1 -- 172.21.15.40:0/3769030368 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25d004e9c0 con 0x7f25d015d180 2023-10-16T08:39:48.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.514+0000 7f25c6ffd640 1 -- 172.21.15.40:0/3769030368 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f25cc0b3380 con 0x7f25d015d180 2023-10-16T08:39:48.530 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.514+0000 7f25c6ffd640 1 --2- 172.21.15.40:0/3769030368 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f25b40664a0 0x7f25b4068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:39:48.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.514+0000 7f25c6ffd640 1 -- 172.21.15.40:0/3769030368 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f25cc11e690 con 0x7f25d015d180 2023-10-16T08:39:48.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.514+0000 7f25d63b3640 1 --2- 172.21.15.40:0/3769030368 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f25b40664a0 0x7f25b4068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:39:48.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.514+0000 7f25d63b3640 1 --2- 172.21.15.40:0/3769030368 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f25b40664a0 0x7f25b4068960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f25c000a970 tx=0x7f25c0008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:39:48.531 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.518+0000 7f25c6ffd640 1 -- 172.21.15.40:0/3769030368 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f25cc0ebbc0 con 0x7f25d015d180 2023-10-16T08:39:48.674 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.666+0000 7f25d8617640 1 -- 172.21.15.40:0/3769030368 --> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f25d00ade70 con 0x7f25b40664a0 2023-10-16T08:39:48.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.694+0000 7f25c6ffd640 1 -- 172.21.15.40:0/3769030368 <== mgr.4111 v2:172.21.15.86:6832/1188970899 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286848 (secure 0 0 0) 0x7f25c000c050 con 0x7f25b40664a0 2023-10-16T08:39:48.698 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:39:48.700 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-16T08:39:48.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.694+0000 7f25d8617640 1 -- 172.21.15.40:0/3769030368 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f25b40664a0 msgr2=0x7f25b4068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:48.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.694+0000 7f25d8617640 1 --2- 172.21.15.40:0/3769030368 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f25b40664a0 0x7f25b4068960 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f25c000a970 tx=0x7f25c0008040 comp rx=0 tx=0).stop 2023-10-16T08:39:48.702 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.694+0000 7f25d8617640 1 -- 172.21.15.40:0/3769030368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d015d180 msgr2=0x7f25d0157c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:39:48.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.694+0000 7f25d8617640 1 --2- 172.21.15.40:0/3769030368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d015d180 0x7f25d0157c50 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f25cc0ad040 tx=0x7f25cc01acd0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.694+0000 7f25d8617640 1 -- 172.21.15.40:0/3769030368 shutdown_connections 2023-10-16T08:39:48.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.694+0000 7f25d8617640 1 --2- 172.21.15.40:0/3769030368 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f25d015d560 0x7f25d0158190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.703 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.694+0000 7f25d8617640 1 --2- 172.21.15.40:0/3769030368 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f25b40664a0 0x7f25b4068960 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.694+0000 7f25d8617640 1 --2- 172.21.15.40:0/3769030368 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f25d015d180 0x7f25d0157c50 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.694+0000 7f25d8617640 1 --2- 172.21.15.40:0/3769030368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f25d00ad820 0x7f25d00c4cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:39:48.704 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.694+0000 7f25d8617640 1 -- 172.21.15.40:0/3769030368 >> 172.21.15.40:0/3769030368 conn(0x7f25d00ce960 msgr2=0x7f25d00ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:39:48.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.694+0000 7f25d8617640 1 -- 172.21.15.40:0/3769030368 shutdown_connections 2023-10-16T08:39:48.705 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:39:48.694+0000 7f25d8617640 1 -- 172.21.15.40:0/3769030368 wait complete. 2023-10-16T08:39:48.717 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":52,"stamp":"2023-10-16T08:39:46.959873+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596433,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":220,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":170,"ondisk_log_size":170,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219200,"kb_used_data":3024,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755520,"statfs":{"total":773094113280,"available":772869652480,"internally_reserved":0,"allocated":3096576,"data_stored":1859362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":141,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004496"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.964601+0000","last_change":"2023-10-16T08:39:26.963087+0000","last_active":"2023-10-16T08:39:26.964601+0000","last_peered":"2023-10-16T08:39:26.964601+0000","last_clean":"2023-10-16T08:39:26.964601+0000","last_became_active":"2023-10-16T08:39:26.962446+0000","last_became_peered":"2023-10-16T08:39:26.962446+0000","last_unstale":"2023-10-16T08:39:26.964601+0000","last_undegraded":"2023-10-16T08:39:26.964601+0000","last_fullsized":"2023-10-16T08:39:26.964601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:33.190218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945446+0000","last_change":"2023-10-16T08:38:45.657659+0000","last_active":"2023-10-16T08:39:25.945446+0000","last_peered":"2023-10-16T08:39:25.945446+0000","last_clean":"2023-10-16T08:39:25.945446+0000","last_became_active":"2023-10-16T08:38:45.656884+0000","last_became_peered":"2023-10-16T08:38:45.656884+0000","last_unstale":"2023-10-16T08:39:25.945446+0000","last_undegraded":"2023-10-16T08:39:25.945446+0000","last_fullsized":"2023-10-16T08:39:25.945446+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402116+0000","last_change":"2023-10-16T08:38:45.630125+0000","last_active":"2023-10-16T08:39:26.402116+0000","last_peered":"2023-10-16T08:39:26.402116+0000","last_clean":"2023-10-16T08:39:26.402116+0000","last_became_active":"2023-10-16T08:38:45.629297+0000","last_became_peered":"2023-10-16T08:38:45.629297+0000","last_unstale":"2023-10-16T08:39:26.402116+0000","last_undegraded":"2023-10-16T08:39:26.402116+0000","last_fullsized":"2023-10-16T08:39:26.402116+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951941+0000","last_change":"2023-10-16T08:38:45.675996+0000","last_active":"2023-10-16T08:39:25.951941+0000","last_peered":"2023-10-16T08:39:25.951941+0000","last_clean":"2023-10-16T08:39:25.951941+0000","last_became_active":"2023-10-16T08:38:45.675327+0000","last_became_peered":"2023-10-16T08:38:45.675327+0000","last_unstale":"2023-10-16T08:39:25.951941+0000","last_undegraded":"2023-10-16T08:39:25.951941+0000","last_fullsized":"2023-10-16T08:39:25.951941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:33.190218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955358+0000","last_change":"2023-10-16T08:38:45.653550+0000","last_active":"2023-10-16T08:39:25.955358+0000","last_peered":"2023-10-16T08:39:25.955358+0000","last_clean":"2023-10-16T08:39:25.955358+0000","last_became_active":"2023-10-16T08:38:45.652808+0000","last_became_peered":"2023-10-16T08:38:45.652808+0000","last_unstale":"2023-10-16T08:39:25.955358+0000","last_undegraded":"2023-10-16T08:39:25.955358+0000","last_fullsized":"2023-10-16T08:39:25.955358+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.950945+0000","last_change":"2023-10-16T08:38:45.675086+0000","last_active":"2023-10-16T08:39:25.950945+0000","last_peered":"2023-10-16T08:39:25.950945+0000","last_clean":"2023-10-16T08:39:25.950945+0000","last_became_active":"2023-10-16T08:38:45.671146+0000","last_became_peered":"2023-10-16T08:38:45.671146+0000","last_unstale":"2023-10-16T08:39:25.950945+0000","last_undegraded":"2023-10-16T08:39:25.950945+0000","last_fullsized":"2023-10-16T08:39:25.950945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954769+0000","last_change":"2023-10-16T08:38:45.652595+0000","last_active":"2023-10-16T08:39:25.954769+0000","last_peered":"2023-10-16T08:39:25.954769+0000","last_clean":"2023-10-16T08:39:25.954769+0000","last_became_active":"2023-10-16T08:38:45.651848+0000","last_became_peered":"2023-10-16T08:38:45.651848+0000","last_unstale":"2023-10-16T08:39:25.954769+0000","last_undegraded":"2023-10-16T08:39:25.954769+0000","last_fullsized":"2023-10-16T08:39:25.954769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:13:13.453321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955228+0000","last_change":"2023-10-16T08:38:45.668858+0000","last_active":"2023-10-16T08:39:25.955228+0000","last_peered":"2023-10-16T08:39:25.955228+0000","last_clean":"2023-10-16T08:39:25.955228+0000","last_became_active":"2023-10-16T08:38:45.664702+0000","last_became_peered":"2023-10-16T08:38:45.664702+0000","last_unstale":"2023-10-16T08:39:25.955228+0000","last_undegraded":"2023-10-16T08:39:25.955228+0000","last_fullsized":"2023-10-16T08:39:25.955228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955879+0000","last_change":"2023-10-16T08:38:45.667037+0000","last_active":"2023-10-16T08:39:25.955879+0000","last_peered":"2023-10-16T08:39:25.955879+0000","last_clean":"2023-10-16T08:39:25.955879+0000","last_became_active":"2023-10-16T08:38:45.666474+0000","last_became_peered":"2023-10-16T08:38:45.666474+0000","last_unstale":"2023-10-16T08:39:25.955879+0000","last_undegraded":"2023-10-16T08:39:25.955879+0000","last_fullsized":"2023-10-16T08:39:25.955879+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:11:02.634006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949980+0000","last_change":"2023-10-16T08:38:45.669859+0000","last_active":"2023-10-16T08:39:25.949980+0000","last_peered":"2023-10-16T08:39:25.949980+0000","last_clean":"2023-10-16T08:39:25.949980+0000","last_became_active":"2023-10-16T08:38:45.668278+0000","last_became_peered":"2023-10-16T08:38:45.668278+0000","last_unstale":"2023-10-16T08:39:25.949980+0000","last_undegraded":"2023-10-16T08:39:25.949980+0000","last_fullsized":"2023-10-16T08:39:25.949980+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949617+0000","last_change":"2023-10-16T08:38:45.664567+0000","last_active":"2023-10-16T08:39:25.949617+0000","last_peered":"2023-10-16T08:39:25.949617+0000","last_clean":"2023-10-16T08:39:25.949617+0000","last_became_active":"2023-10-16T08:38:45.663813+0000","last_became_peered":"2023-10-16T08:38:45.663813+0000","last_unstale":"2023-10-16T08:39:25.949617+0000","last_undegraded":"2023-10-16T08:39:25.949617+0000","last_fullsized":"2023-10-16T08:39:25.949617+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403718+0000","last_change":"2023-10-16T08:38:45.675925+0000","last_active":"2023-10-16T08:39:26.403718+0000","last_peered":"2023-10-16T08:39:26.403718+0000","last_clean":"2023-10-16T08:39:26.403718+0000","last_became_active":"2023-10-16T08:38:45.675320+0000","last_became_peered":"2023-10-16T08:38:45.675320+0000","last_unstale":"2023-10-16T08:39:26.403718+0000","last_undegraded":"2023-10-16T08:39:26.403718+0000","last_fullsized":"2023-10-16T08:39:26.403718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:11.991837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956028+0000","last_change":"2023-10-16T08:38:45.668056+0000","last_active":"2023-10-16T08:39:25.956028+0000","last_peered":"2023-10-16T08:39:25.956028+0000","last_clean":"2023-10-16T08:39:25.956028+0000","last_became_active":"2023-10-16T08:38:45.667428+0000","last_became_peered":"2023-10-16T08:38:45.667428+0000","last_unstale":"2023-10-16T08:39:25.956028+0000","last_undegraded":"2023-10-16T08:39:25.956028+0000","last_fullsized":"2023-10-16T08:39:25.956028+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.401718+0000","last_change":"2023-10-16T08:38:45.637518+0000","last_active":"2023-10-16T08:39:26.401718+0000","last_peered":"2023-10-16T08:39:26.401718+0000","last_clean":"2023-10-16T08:39:26.401718+0000","last_became_active":"2023-10-16T08:38:45.636785+0000","last_became_peered":"2023-10-16T08:38:45.636785+0000","last_unstale":"2023-10-16T08:39:26.401718+0000","last_undegraded":"2023-10-16T08:39:26.401718+0000","last_fullsized":"2023-10-16T08:39:26.401718+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:11.870171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.944872+0000","last_change":"2023-10-16T08:38:45.656433+0000","last_active":"2023-10-16T08:39:25.944872+0000","last_peered":"2023-10-16T08:39:25.944872+0000","last_clean":"2023-10-16T08:39:25.944872+0000","last_became_active":"2023-10-16T08:38:45.655395+0000","last_became_peered":"2023-10-16T08:38:45.655395+0000","last_unstale":"2023-10-16T08:39:25.944872+0000","last_undegraded":"2023-10-16T08:39:25.944872+0000","last_fullsized":"2023-10-16T08:39:25.944872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.950540+0000","last_change":"2023-10-16T08:38:45.664192+0000","last_active":"2023-10-16T08:39:25.950540+0000","last_peered":"2023-10-16T08:39:25.950540+0000","last_clean":"2023-10-16T08:39:25.950540+0000","last_became_active":"2023-10-16T08:38:45.663312+0000","last_became_peered":"2023-10-16T08:38:45.663312+0000","last_unstale":"2023-10-16T08:39:25.950540+0000","last_undegraded":"2023-10-16T08:39:25.950540+0000","last_fullsized":"2023-10-16T08:39:25.950540+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.016191+0000","last_change":"2023-10-16T08:39:27.013093+0000","last_active":"2023-10-16T08:39:27.016191+0000","last_peered":"2023-10-16T08:39:27.016191+0000","last_clean":"2023-10-16T08:39:27.016191+0000","last_became_active":"2023-10-16T08:39:27.012068+0000","last_became_peered":"2023-10-16T08:39:27.012068+0000","last_unstale":"2023-10-16T08:39:27.016191+0000","last_undegraded":"2023-10-16T08:39:27.016191+0000","last_fullsized":"2023-10-16T08:39:27.016191+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945605+0000","last_change":"2023-10-16T08:38:45.631488+0000","last_active":"2023-10-16T08:39:25.945605+0000","last_peered":"2023-10-16T08:39:25.945605+0000","last_clean":"2023-10-16T08:39:25.945605+0000","last_became_active":"2023-10-16T08:38:45.630641+0000","last_became_peered":"2023-10-16T08:38:45.630641+0000","last_unstale":"2023-10-16T08:39:25.945605+0000","last_undegraded":"2023-10-16T08:39:25.945605+0000","last_fullsized":"2023-10-16T08:39:25.945605+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:11.991837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955988+0000","last_change":"2023-10-16T08:38:45.650087+0000","last_active":"2023-10-16T08:39:25.955988+0000","last_peered":"2023-10-16T08:39:25.955988+0000","last_clean":"2023-10-16T08:39:25.955988+0000","last_became_active":"2023-10-16T08:38:45.649475+0000","last_became_peered":"2023-10-16T08:38:45.649475+0000","last_unstale":"2023-10-16T08:39:25.955988+0000","last_undegraded":"2023-10-16T08:39:25.955988+0000","last_fullsized":"2023-10-16T08:39:25.955988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:26.292137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949532+0000","last_change":"2023-10-16T08:38:45.667336+0000","last_active":"2023-10-16T08:39:25.949532+0000","last_peered":"2023-10-16T08:39:25.949532+0000","last_clean":"2023-10-16T08:39:25.949532+0000","last_became_active":"2023-10-16T08:38:45.665291+0000","last_became_peered":"2023-10-16T08:38:45.665291+0000","last_unstale":"2023-10-16T08:39:25.949532+0000","last_undegraded":"2023-10-16T08:39:25.949532+0000","last_fullsized":"2023-10-16T08:39:25.949532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402191+0000","last_change":"2023-10-16T08:38:45.654036+0000","last_active":"2023-10-16T08:39:26.402191+0000","last_peered":"2023-10-16T08:39:26.402191+0000","last_clean":"2023-10-16T08:39:26.402191+0000","last_became_active":"2023-10-16T08:38:45.652412+0000","last_became_peered":"2023-10-16T08:38:45.652412+0000","last_unstale":"2023-10-16T08:39:26.402191+0000","last_undegraded":"2023-10-16T08:39:26.402191+0000","last_fullsized":"2023-10-16T08:39:26.402191+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951156+0000","last_change":"2023-10-16T08:38:45.677644+0000","last_active":"2023-10-16T08:39:25.951156+0000","last_peered":"2023-10-16T08:39:25.951156+0000","last_clean":"2023-10-16T08:39:25.951156+0000","last_became_active":"2023-10-16T08:38:45.676844+0000","last_became_peered":"2023-10-16T08:38:45.676844+0000","last_unstale":"2023-10-16T08:39:25.951156+0000","last_undegraded":"2023-10-16T08:39:25.951156+0000","last_fullsized":"2023-10-16T08:39:25.951156+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955256+0000","last_change":"2023-10-16T08:38:45.663771+0000","last_active":"2023-10-16T08:39:25.955256+0000","last_peered":"2023-10-16T08:39:25.955256+0000","last_clean":"2023-10-16T08:39:25.955256+0000","last_became_active":"2023-10-16T08:38:45.662944+0000","last_became_peered":"2023-10-16T08:38:45.662944+0000","last_unstale":"2023-10-16T08:39:25.955256+0000","last_undegraded":"2023-10-16T08:39:25.955256+0000","last_fullsized":"2023-10-16T08:39:25.955256+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.950669+0000","last_change":"2023-10-16T08:38:45.667398+0000","last_active":"2023-10-16T08:39:25.950669+0000","last_peered":"2023-10-16T08:39:25.950669+0000","last_clean":"2023-10-16T08:39:25.950669+0000","last_became_active":"2023-10-16T08:38:45.666641+0000","last_became_peered":"2023-10-16T08:38:45.666641+0000","last_unstale":"2023-10-16T08:39:25.950669+0000","last_undegraded":"2023-10-16T08:39:25.950669+0000","last_fullsized":"2023-10-16T08:39:25.950669+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955685+0000","last_change":"2023-10-16T08:38:45.670526+0000","last_active":"2023-10-16T08:39:25.955685+0000","last_peered":"2023-10-16T08:39:25.955685+0000","last_clean":"2023-10-16T08:39:25.955685+0000","last_became_active":"2023-10-16T08:38:45.669734+0000","last_became_peered":"2023-10-16T08:38:45.669734+0000","last_unstale":"2023-10-16T08:39:25.955685+0000","last_undegraded":"2023-10-16T08:39:25.955685+0000","last_fullsized":"2023-10-16T08:39:25.955685+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949999+0000","last_change":"2023-10-16T08:38:45.670429+0000","last_active":"2023-10-16T08:39:25.949999+0000","last_peered":"2023-10-16T08:39:25.949999+0000","last_clean":"2023-10-16T08:39:25.949999+0000","last_became_active":"2023-10-16T08:38:45.665972+0000","last_became_peered":"2023-10-16T08:38:45.665972+0000","last_unstale":"2023-10-16T08:39:25.949999+0000","last_undegraded":"2023-10-16T08:39:25.949999+0000","last_fullsized":"2023-10-16T08:39:25.949999+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945074+0000","last_change":"2023-10-16T08:38:45.667047+0000","last_active":"2023-10-16T08:39:25.945074+0000","last_peered":"2023-10-16T08:39:25.945074+0000","last_clean":"2023-10-16T08:39:25.945074+0000","last_became_active":"2023-10-16T08:38:45.666403+0000","last_became_peered":"2023-10-16T08:38:45.666403+0000","last_unstale":"2023-10-16T08:39:25.945074+0000","last_undegraded":"2023-10-16T08:39:25.945074+0000","last_fullsized":"2023-10-16T08:39:25.945074+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956047+0000","last_change":"2023-10-16T08:38:45.668007+0000","last_active":"2023-10-16T08:39:25.956047+0000","last_peered":"2023-10-16T08:39:25.956047+0000","last_clean":"2023-10-16T08:39:25.956047+0000","last_became_active":"2023-10-16T08:38:45.664572+0000","last_became_peered":"2023-10-16T08:38:45.664572+0000","last_unstale":"2023-10-16T08:39:25.956047+0000","last_undegraded":"2023-10-16T08:39:25.956047+0000","last_fullsized":"2023-10-16T08:39:25.956047+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402369+0000","last_change":"2023-10-16T08:38:45.667378+0000","last_active":"2023-10-16T08:39:26.402369+0000","last_peered":"2023-10-16T08:39:26.402369+0000","last_clean":"2023-10-16T08:39:26.402369+0000","last_became_active":"2023-10-16T08:38:45.666547+0000","last_became_peered":"2023-10-16T08:38:45.666547+0000","last_unstale":"2023-10-16T08:39:26.402369+0000","last_undegraded":"2023-10-16T08:39:26.402369+0000","last_fullsized":"2023-10-16T08:39:26.402369+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.013174+0000","last_change":"2023-10-16T08:39:27.010902+0000","last_active":"2023-10-16T08:39:27.013174+0000","last_peered":"2023-10-16T08:39:27.013174+0000","last_clean":"2023-10-16T08:39:27.013174+0000","last_became_active":"2023-10-16T08:39:27.009983+0000","last_became_peered":"2023-10-16T08:39:27.009983+0000","last_unstale":"2023-10-16T08:39:27.013174+0000","last_undegraded":"2023-10-16T08:39:27.013174+0000","last_fullsized":"2023-10-16T08:39:27.013174+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:45:19.823464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.014469+0000","last_change":"2023-10-16T08:39:27.014469+0000","last_active":"2023-10-16T08:39:27.014469+0000","last_peered":"2023-10-16T08:39:27.014469+0000","last_clean":"2023-10-16T08:39:27.014469+0000","last_became_active":"2023-10-16T08:39:27.013658+0000","last_became_peered":"2023-10-16T08:39:27.013658+0000","last_unstale":"2023-10-16T08:39:27.014469+0000","last_undegraded":"2023-10-16T08:39:27.014469+0000","last_fullsized":"2023-10-16T08:39:27.014469+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949746+0000","last_change":"2023-10-16T08:38:45.638183+0000","last_active":"2023-10-16T08:39:25.949746+0000","last_peered":"2023-10-16T08:39:25.949746+0000","last_clean":"2023-10-16T08:39:25.949746+0000","last_became_active":"2023-10-16T08:38:45.636762+0000","last_became_peered":"2023-10-16T08:38:45.636762+0000","last_unstale":"2023-10-16T08:39:25.949746+0000","last_undegraded":"2023-10-16T08:39:25.949746+0000","last_fullsized":"2023-10-16T08:39:25.949746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403351+0000","last_change":"2023-10-16T08:38:45.659464+0000","last_active":"2023-10-16T08:39:26.403351+0000","last_peered":"2023-10-16T08:39:26.403351+0000","last_clean":"2023-10-16T08:39:26.403351+0000","last_became_active":"2023-10-16T08:38:45.658808+0000","last_became_peered":"2023-10-16T08:38:45.658808+0000","last_unstale":"2023-10-16T08:39:26.403351+0000","last_undegraded":"2023-10-16T08:39:26.403351+0000","last_fullsized":"2023-10-16T08:39:26.403351+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:25.386221+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956227+0000","last_change":"2023-10-16T08:38:44.644122+0000","last_active":"2023-10-16T08:39:25.956227+0000","last_peered":"2023-10-16T08:39:25.956227+0000","last_clean":"2023-10-16T08:39:25.956227+0000","last_became_active":"2023-10-16T08:38:44.643614+0000","last_became_peered":"2023-10-16T08:38:44.643614+0000","last_unstale":"2023-10-16T08:39:25.956227+0000","last_undegraded":"2023-10-16T08:39:25.956227+0000","last_fullsized":"2023-10-16T08:39:25.956227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402402+0000","last_change":"2023-10-16T08:38:45.659854+0000","last_active":"2023-10-16T08:39:26.402402+0000","last_peered":"2023-10-16T08:39:26.402402+0000","last_clean":"2023-10-16T08:39:26.402402+0000","last_became_active":"2023-10-16T08:38:45.658409+0000","last_became_peered":"2023-10-16T08:38:45.658409+0000","last_unstale":"2023-10-16T08:39:26.402402+0000","last_undegraded":"2023-10-16T08:39:26.402402+0000","last_fullsized":"2023-10-16T08:39:26.402402+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949532+0000","last_change":"2023-10-16T08:38:44.658819+0000","last_active":"2023-10-16T08:39:25.949532+0000","last_peered":"2023-10-16T08:39:25.949532+0000","last_clean":"2023-10-16T08:39:25.949532+0000","last_became_active":"2023-10-16T08:38:44.658257+0000","last_became_peered":"2023-10-16T08:38:44.658257+0000","last_unstale":"2023-10-16T08:39:25.949532+0000","last_undegraded":"2023-10-16T08:39:25.949532+0000","last_fullsized":"2023-10-16T08:39:25.949532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:54.848445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955337+0000","last_change":"2023-10-16T08:38:44.647536+0000","last_active":"2023-10-16T08:39:25.955337+0000","last_peered":"2023-10-16T08:39:25.955337+0000","last_clean":"2023-10-16T08:39:25.955337+0000","last_became_active":"2023-10-16T08:38:44.645931+0000","last_became_peered":"2023-10-16T08:38:44.645931+0000","last_unstale":"2023-10-16T08:39:25.955337+0000","last_undegraded":"2023-10-16T08:39:25.955337+0000","last_fullsized":"2023-10-16T08:39:25.955337+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955491+0000","last_change":"2023-10-16T08:38:45.662855+0000","last_active":"2023-10-16T08:39:25.955491+0000","last_peered":"2023-10-16T08:39:25.955491+0000","last_clean":"2023-10-16T08:39:25.955491+0000","last_became_active":"2023-10-16T08:38:45.662072+0000","last_became_peered":"2023-10-16T08:38:45.662072+0000","last_unstale":"2023-10-16T08:39:25.955491+0000","last_undegraded":"2023-10-16T08:39:25.955491+0000","last_fullsized":"2023-10-16T08:39:25.955491+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.950636+0000","last_change":"2023-10-16T08:38:44.657651+0000","last_active":"2023-10-16T08:39:25.950636+0000","last_peered":"2023-10-16T08:39:25.950636+0000","last_clean":"2023-10-16T08:39:25.950636+0000","last_became_active":"2023-10-16T08:38:44.657065+0000","last_became_peered":"2023-10-16T08:38:44.657065+0000","last_unstale":"2023-10-16T08:39:25.950636+0000","last_undegraded":"2023-10-16T08:39:25.950636+0000","last_fullsized":"2023-10-16T08:39:25.950636+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.944769+0000","last_change":"2023-10-16T08:38:45.668048+0000","last_active":"2023-10-16T08:39:25.944769+0000","last_peered":"2023-10-16T08:39:25.944769+0000","last_clean":"2023-10-16T08:39:25.944769+0000","last_became_active":"2023-10-16T08:38:45.667413+0000","last_became_peered":"2023-10-16T08:38:45.667413+0000","last_unstale":"2023-10-16T08:39:25.944769+0000","last_undegraded":"2023-10-16T08:39:25.944769+0000","last_fullsized":"2023-10-16T08:39:25.944769+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949842+0000","last_change":"2023-10-16T08:38:44.663188+0000","last_active":"2023-10-16T08:39:25.949842+0000","last_peered":"2023-10-16T08:39:25.949842+0000","last_clean":"2023-10-16T08:39:25.949842+0000","last_became_active":"2023-10-16T08:38:44.662383+0000","last_became_peered":"2023-10-16T08:38:44.662383+0000","last_unstale":"2023-10-16T08:39:25.949842+0000","last_undegraded":"2023-10-16T08:39:25.949842+0000","last_fullsized":"2023-10-16T08:39:25.949842+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:24.375366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.953576+0000","last_change":"2023-10-16T08:38:45.679271+0000","last_active":"2023-10-16T08:39:25.953576+0000","last_peered":"2023-10-16T08:39:25.953576+0000","last_clean":"2023-10-16T08:39:25.953576+0000","last_became_active":"2023-10-16T08:38:45.678050+0000","last_became_peered":"2023-10-16T08:38:45.678050+0000","last_unstale":"2023-10-16T08:39:25.953576+0000","last_undegraded":"2023-10-16T08:39:25.953576+0000","last_fullsized":"2023-10-16T08:39:25.953576+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.953752+0000","last_change":"2023-10-16T08:38:44.653478+0000","last_active":"2023-10-16T08:39:25.953752+0000","last_peered":"2023-10-16T08:39:25.953752+0000","last_clean":"2023-10-16T08:39:25.953752+0000","last_became_active":"2023-10-16T08:38:44.652618+0000","last_became_peered":"2023-10-16T08:38:44.652618+0000","last_unstale":"2023-10-16T08:39:25.953752+0000","last_undegraded":"2023-10-16T08:39:25.953752+0000","last_fullsized":"2023-10-16T08:39:25.953752+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402370+0000","last_change":"2023-10-16T08:38:45.656603+0000","last_active":"2023-10-16T08:39:26.402370+0000","last_peered":"2023-10-16T08:39:26.402370+0000","last_clean":"2023-10-16T08:39:26.402370+0000","last_became_active":"2023-10-16T08:38:45.655213+0000","last_became_peered":"2023-10-16T08:38:45.655213+0000","last_unstale":"2023-10-16T08:39:26.402370+0000","last_undegraded":"2023-10-16T08:39:26.402370+0000","last_fullsized":"2023-10-16T08:39:26.402370+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":17,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945084+0000","last_change":"2023-10-16T08:38:44.651637+0000","last_active":"2023-10-16T08:39:25.945084+0000","last_peered":"2023-10-16T08:39:25.945084+0000","last_clean":"2023-10-16T08:39:25.945084+0000","last_became_active":"2023-10-16T08:38:44.651359+0000","last_became_peered":"2023-10-16T08:38:44.651359+0000","last_unstale":"2023-10-16T08:39:25.945084+0000","last_undegraded":"2023-10-16T08:39:25.945084+0000","last_fullsized":"2023-10-16T08:39:25.945084+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:53:10.859316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403856+0000","last_change":"2023-10-16T08:38:45.661741+0000","last_active":"2023-10-16T08:39:26.403856+0000","last_peered":"2023-10-16T08:39:26.403856+0000","last_clean":"2023-10-16T08:39:26.403856+0000","last_became_active":"2023-10-16T08:38:45.660611+0000","last_became_peered":"2023-10-16T08:38:45.660611+0000","last_unstale":"2023-10-16T08:39:26.403856+0000","last_undegraded":"2023-10-16T08:39:26.403856+0000","last_fullsized":"2023-10-16T08:39:26.403856+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:55.859300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'18","reported_seq":32,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:35.375472+0000","last_change":"2023-10-16T08:38:44.647593+0000","last_active":"2023-10-16T08:39:35.375472+0000","last_peered":"2023-10-16T08:39:35.375472+0000","last_clean":"2023-10-16T08:39:35.375472+0000","last_became_active":"2023-10-16T08:38:44.647001+0000","last_became_peered":"2023-10-16T08:38:44.647001+0000","last_unstale":"2023-10-16T08:39:35.375472+0000","last_undegraded":"2023-10-16T08:39:35.375472+0000","last_fullsized":"2023-10-16T08:39:35.375472+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5267,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955477+0000","last_change":"2023-10-16T08:38:45.668629+0000","last_active":"2023-10-16T08:39:25.955477+0000","last_peered":"2023-10-16T08:39:25.955477+0000","last_clean":"2023-10-16T08:39:25.955477+0000","last_became_active":"2023-10-16T08:38:45.667334+0000","last_became_peered":"2023-10-16T08:38:45.667334+0000","last_unstale":"2023-10-16T08:39:25.955477+0000","last_undegraded":"2023-10-16T08:39:25.955477+0000","last_fullsized":"2023-10-16T08:39:25.955477+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:14:16.284469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402610+0000","last_change":"2023-10-16T08:38:44.655832+0000","last_active":"2023-10-16T08:39:26.402610+0000","last_peered":"2023-10-16T08:39:26.402610+0000","last_clean":"2023-10-16T08:39:26.402610+0000","last_became_active":"2023-10-16T08:38:44.655469+0000","last_became_peered":"2023-10-16T08:38:44.655469+0000","last_unstale":"2023-10-16T08:39:26.402610+0000","last_undegraded":"2023-10-16T08:39:26.402610+0000","last_fullsized":"2023-10-16T08:39:26.402610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:19.768762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.401855+0000","last_change":"2023-10-16T08:38:45.653688+0000","last_active":"2023-10-16T08:39:26.401855+0000","last_peered":"2023-10-16T08:39:26.401855+0000","last_clean":"2023-10-16T08:39:26.401855+0000","last_became_active":"2023-10-16T08:38:45.650563+0000","last_became_peered":"2023-10-16T08:38:45.650563+0000","last_unstale":"2023-10-16T08:39:26.401855+0000","last_undegraded":"2023-10-16T08:39:26.401855+0000","last_fullsized":"2023-10-16T08:39:26.401855+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:29.539285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951818+0000","last_change":"2023-10-16T08:38:44.662816+0000","last_active":"2023-10-16T08:39:25.951818+0000","last_peered":"2023-10-16T08:39:25.951818+0000","last_clean":"2023-10-16T08:39:25.951818+0000","last_became_active":"2023-10-16T08:38:44.662110+0000","last_became_peered":"2023-10-16T08:38:44.662110+0000","last_unstale":"2023-10-16T08:39:25.951818+0000","last_undegraded":"2023-10-16T08:39:25.951818+0000","last_fullsized":"2023-10-16T08:39:25.951818+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:00.201724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955811+0000","last_change":"2023-10-16T08:38:45.662166+0000","last_active":"2023-10-16T08:39:25.955811+0000","last_peered":"2023-10-16T08:39:25.955811+0000","last_clean":"2023-10-16T08:39:25.955811+0000","last_became_active":"2023-10-16T08:38:45.661595+0000","last_became_peered":"2023-10-16T08:38:45.661595+0000","last_unstale":"2023-10-16T08:39:25.955811+0000","last_undegraded":"2023-10-16T08:39:25.955811+0000","last_fullsized":"2023-10-16T08:39:25.955811+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:02:39.840175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949503+0000","last_change":"2023-10-16T08:38:44.657766+0000","last_active":"2023-10-16T08:39:25.949503+0000","last_peered":"2023-10-16T08:39:25.949503+0000","last_clean":"2023-10-16T08:39:25.949503+0000","last_became_active":"2023-10-16T08:38:44.657153+0000","last_became_peered":"2023-10-16T08:38:44.657153+0000","last_unstale":"2023-10-16T08:39:25.949503+0000","last_undegraded":"2023-10-16T08:39:25.949503+0000","last_fullsized":"2023-10-16T08:39:25.949503+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949501+0000","last_change":"2023-10-16T08:38:45.637591+0000","last_active":"2023-10-16T08:39:25.949501+0000","last_peered":"2023-10-16T08:39:25.949501+0000","last_clean":"2023-10-16T08:39:25.949501+0000","last_became_active":"2023-10-16T08:38:45.635633+0000","last_became_peered":"2023-10-16T08:38:45.635633+0000","last_unstale":"2023-10-16T08:39:25.949501+0000","last_undegraded":"2023-10-16T08:39:25.949501+0000","last_fullsized":"2023-10-16T08:39:25.949501+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:23.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"19'8","reported_seq":22,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.946160+0000","last_change":"2023-10-16T08:38:44.638297+0000","last_active":"2023-10-16T08:39:25.946160+0000","last_peered":"2023-10-16T08:39:25.946160+0000","last_clean":"2023-10-16T08:39:25.946160+0000","last_became_active":"2023-10-16T08:38:44.637640+0000","last_became_peered":"2023-10-16T08:38:44.637640+0000","last_unstale":"2023-10-16T08:39:25.946160+0000","last_undegraded":"2023-10-16T08:39:25.946160+0000","last_fullsized":"2023-10-16T08:39:25.946160+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:16.003943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954941+0000","last_change":"2023-10-16T08:38:45.657286+0000","last_active":"2023-10-16T08:39:25.954941+0000","last_peered":"2023-10-16T08:39:25.954941+0000","last_clean":"2023-10-16T08:39:25.954941+0000","last_became_active":"2023-10-16T08:38:45.656651+0000","last_became_peered":"2023-10-16T08:38:45.656651+0000","last_unstale":"2023-10-16T08:39:25.954941+0000","last_undegraded":"2023-10-16T08:39:25.954941+0000","last_fullsized":"2023-10-16T08:39:25.954941+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:28.052483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403973+0000","last_change":"2023-10-16T08:38:44.648434+0000","last_active":"2023-10-16T08:39:26.403973+0000","last_peered":"2023-10-16T08:39:26.403973+0000","last_clean":"2023-10-16T08:39:26.403973+0000","last_became_active":"2023-10-16T08:38:44.647887+0000","last_became_peered":"2023-10-16T08:38:44.647887+0000","last_unstale":"2023-10-16T08:39:26.403973+0000","last_undegraded":"2023-10-16T08:39:26.403973+0000","last_fullsized":"2023-10-16T08:39:26.403973+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954988+0000","last_change":"2023-10-16T08:38:45.661965+0000","last_active":"2023-10-16T08:39:25.954988+0000","last_peered":"2023-10-16T08:39:25.954988+0000","last_clean":"2023-10-16T08:39:25.954988+0000","last_became_active":"2023-10-16T08:38:45.656750+0000","last_became_peered":"2023-10-16T08:38:45.656750+0000","last_unstale":"2023-10-16T08:39:25.954988+0000","last_undegraded":"2023-10-16T08:39:25.954988+0000","last_fullsized":"2023-10-16T08:39:25.954988+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945809+0000","last_change":"2023-10-16T08:38:44.647524+0000","last_active":"2023-10-16T08:39:25.945809+0000","last_peered":"2023-10-16T08:39:25.945809+0000","last_clean":"2023-10-16T08:39:25.945809+0000","last_became_active":"2023-10-16T08:38:44.646042+0000","last_became_peered":"2023-10-16T08:38:44.646042+0000","last_unstale":"2023-10-16T08:39:25.945809+0000","last_undegraded":"2023-10-16T08:39:25.945809+0000","last_fullsized":"2023-10-16T08:39:25.945809+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:11:24.375366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955916+0000","last_change":"2023-10-16T08:38:45.670969+0000","last_active":"2023-10-16T08:39:25.955916+0000","last_peered":"2023-10-16T08:39:25.955916+0000","last_clean":"2023-10-16T08:39:25.955916+0000","last_became_active":"2023-10-16T08:38:45.670415+0000","last_became_peered":"2023-10-16T08:38:45.670415+0000","last_unstale":"2023-10-16T08:39:25.955916+0000","last_undegraded":"2023-10-16T08:39:25.955916+0000","last_fullsized":"2023-10-16T08:39:25.955916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:15:00.472153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945149+0000","last_change":"2023-10-16T08:38:44.655744+0000","last_active":"2023-10-16T08:39:25.945149+0000","last_peered":"2023-10-16T08:39:25.945149+0000","last_clean":"2023-10-16T08:39:25.945149+0000","last_became_active":"2023-10-16T08:38:44.655260+0000","last_became_peered":"2023-10-16T08:38:44.655260+0000","last_unstale":"2023-10-16T08:39:25.945149+0000","last_undegraded":"2023-10-16T08:39:25.945149+0000","last_fullsized":"2023-10-16T08:39:25.945149+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:50:54.848445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951801+0000","last_change":"2023-10-16T08:38:45.668230+0000","last_active":"2023-10-16T08:39:25.951801+0000","last_peered":"2023-10-16T08:39:25.951801+0000","last_clean":"2023-10-16T08:39:25.951801+0000","last_became_active":"2023-10-16T08:38:45.667627+0000","last_became_peered":"2023-10-16T08:38:45.667627+0000","last_unstale":"2023-10-16T08:39:25.951801+0000","last_undegraded":"2023-10-16T08:39:25.951801+0000","last_fullsized":"2023-10-16T08:39:25.951801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:28:28.052483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945644+0000","last_change":"2023-10-16T08:38:44.662118+0000","last_active":"2023-10-16T08:39:25.945644+0000","last_peered":"2023-10-16T08:39:25.945644+0000","last_clean":"2023-10-16T08:39:25.945644+0000","last_became_active":"2023-10-16T08:38:44.661736+0000","last_became_peered":"2023-10-16T08:38:44.661736+0000","last_unstale":"2023-10-16T08:39:25.945644+0000","last_undegraded":"2023-10-16T08:39:25.945644+0000","last_fullsized":"2023-10-16T08:39:25.945644+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:49:40.967349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955744+0000","last_change":"2023-10-16T08:38:45.667423+0000","last_active":"2023-10-16T08:39:25.955744+0000","last_peered":"2023-10-16T08:39:25.955744+0000","last_clean":"2023-10-16T08:39:25.955744+0000","last_became_active":"2023-10-16T08:38:45.666390+0000","last_became_peered":"2023-10-16T08:38:45.666390+0000","last_unstale":"2023-10-16T08:39:25.955744+0000","last_undegraded":"2023-10-16T08:39:25.955744+0000","last_fullsized":"2023-10-16T08:39:25.955744+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:13:33.190218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":167,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954612+0000","last_change":"2023-10-16T08:38:34.313511+0000","last_active":"2023-10-16T08:39:25.954612+0000","last_peered":"2023-10-16T08:39:25.954612+0000","last_clean":"2023-10-16T08:39:25.954612+0000","last_became_active":"2023-10-16T08:38:34.310944+0000","last_became_peered":"2023-10-16T08:38:34.310944+0000","last_unstale":"2023-10-16T08:39:25.954612+0000","last_undegraded":"2023-10-16T08:39:25.954612+0000","last_fullsized":"2023-10-16T08:39:25.954612+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:33.264692+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:33.264692+0000","last_clean_scrub_stamp":"2023-10-16T08:38:33.264692+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:37:50.170461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951158+0000","last_change":"2023-10-16T08:38:44.655223+0000","last_active":"2023-10-16T08:39:25.951158+0000","last_peered":"2023-10-16T08:39:25.951158+0000","last_clean":"2023-10-16T08:39:25.951158+0000","last_became_active":"2023-10-16T08:38:44.654277+0000","last_became_peered":"2023-10-16T08:38:44.654277+0000","last_unstale":"2023-10-16T08:39:25.951158+0000","last_undegraded":"2023-10-16T08:39:25.951158+0000","last_fullsized":"2023-10-16T08:39:25.951158+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:03.462433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955641+0000","last_change":"2023-10-16T08:38:45.679868+0000","last_active":"2023-10-16T08:39:25.955641+0000","last_peered":"2023-10-16T08:39:25.955641+0000","last_clean":"2023-10-16T08:39:25.955641+0000","last_became_active":"2023-10-16T08:38:45.679409+0000","last_became_peered":"2023-10-16T08:38:45.679409+0000","last_unstale":"2023-10-16T08:39:25.955641+0000","last_undegraded":"2023-10-16T08:39:25.955641+0000","last_fullsized":"2023-10-16T08:39:25.955641+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:47.591986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954528+0000","last_change":"2023-10-16T08:38:44.652152+0000","last_active":"2023-10-16T08:39:25.954528+0000","last_peered":"2023-10-16T08:39:25.954528+0000","last_clean":"2023-10-16T08:39:25.954528+0000","last_became_active":"2023-10-16T08:38:44.651534+0000","last_became_peered":"2023-10-16T08:38:44.651534+0000","last_unstale":"2023-10-16T08:39:25.954528+0000","last_undegraded":"2023-10-16T08:39:25.954528+0000","last_fullsized":"2023-10-16T08:39:25.954528+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:09:31.060880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.401719+0000","last_change":"2023-10-16T08:38:45.651786+0000","last_active":"2023-10-16T08:39:26.401719+0000","last_peered":"2023-10-16T08:39:26.401719+0000","last_clean":"2023-10-16T08:39:26.401719+0000","last_became_active":"2023-10-16T08:38:45.650715+0000","last_became_peered":"2023-10-16T08:38:45.650715+0000","last_unstale":"2023-10-16T08:39:26.401719+0000","last_undegraded":"2023-10-16T08:39:26.401719+0000","last_fullsized":"2023-10-16T08:39:26.401719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:29.799799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.946109+0000","last_change":"2023-10-16T08:38:44.645933+0000","last_active":"2023-10-16T08:39:25.946109+0000","last_peered":"2023-10-16T08:39:25.946109+0000","last_clean":"2023-10-16T08:39:25.946109+0000","last_became_active":"2023-10-16T08:38:44.645451+0000","last_became_peered":"2023-10-16T08:38:44.645451+0000","last_unstale":"2023-10-16T08:39:25.946109+0000","last_undegraded":"2023-10-16T08:39:25.946109+0000","last_fullsized":"2023-10-16T08:39:25.946109+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.944780+0000","last_change":"2023-10-16T08:38:45.641499+0000","last_active":"2023-10-16T08:39:25.944780+0000","last_peered":"2023-10-16T08:39:25.944780+0000","last_clean":"2023-10-16T08:39:25.944780+0000","last_became_active":"2023-10-16T08:38:45.640765+0000","last_became_peered":"2023-10-16T08:38:45.640765+0000","last_unstale":"2023-10-16T08:39:25.944780+0000","last_undegraded":"2023-10-16T08:39:25.944780+0000","last_fullsized":"2023-10-16T08:39:25.944780+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:57:04.473288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.952050+0000","last_change":"2023-10-16T08:38:44.655401+0000","last_active":"2023-10-16T08:39:25.952050+0000","last_peered":"2023-10-16T08:39:25.952050+0000","last_clean":"2023-10-16T08:39:25.952050+0000","last_became_active":"2023-10-16T08:38:44.654622+0000","last_became_peered":"2023-10-16T08:38:44.654622+0000","last_unstale":"2023-10-16T08:39:25.952050+0000","last_undegraded":"2023-10-16T08:39:25.952050+0000","last_fullsized":"2023-10-16T08:39:25.952050+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:48.705915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403580+0000","last_change":"2023-10-16T08:38:45.659377+0000","last_active":"2023-10-16T08:39:26.403580+0000","last_peered":"2023-10-16T08:39:26.403580+0000","last_clean":"2023-10-16T08:39:26.403580+0000","last_became_active":"2023-10-16T08:38:45.658594+0000","last_became_peered":"2023-10-16T08:38:45.658594+0000","last_unstale":"2023-10-16T08:39:26.403580+0000","last_undegraded":"2023-10-16T08:39:26.403580+0000","last_fullsized":"2023-10-16T08:39:26.403580+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:56.738058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945087+0000","last_change":"2023-10-16T08:38:44.634158+0000","last_active":"2023-10-16T08:39:25.945087+0000","last_peered":"2023-10-16T08:39:25.945087+0000","last_clean":"2023-10-16T08:39:25.945087+0000","last_became_active":"2023-10-16T08:38:44.633854+0000","last_became_peered":"2023-10-16T08:38:44.633854+0000","last_unstale":"2023-10-16T08:39:25.945087+0000","last_undegraded":"2023-10-16T08:39:25.945087+0000","last_fullsized":"2023-10-16T08:39:25.945087+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:25:55.727203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.015974+0000","last_change":"2023-10-16T08:39:27.015974+0000","last_active":"2023-10-16T08:39:27.015974+0000","last_peered":"2023-10-16T08:39:27.015974+0000","last_clean":"2023-10-16T08:39:27.015974+0000","last_became_active":"2023-10-16T08:39:27.015266+0000","last_became_peered":"2023-10-16T08:39:27.015266+0000","last_unstale":"2023-10-16T08:39:27.015974+0000","last_undegraded":"2023-10-16T08:39:27.015974+0000","last_fullsized":"2023-10-16T08:39:27.015974+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:41:52.951028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956207+0000","last_change":"2023-10-16T08:38:44.655693+0000","last_active":"2023-10-16T08:39:25.956207+0000","last_peered":"2023-10-16T08:39:25.956207+0000","last_clean":"2023-10-16T08:39:25.956207+0000","last_became_active":"2023-10-16T08:38:44.655001+0000","last_became_peered":"2023-10-16T08:38:44.655001+0000","last_unstale":"2023-10-16T08:39:25.956207+0000","last_undegraded":"2023-10-16T08:39:25.956207+0000","last_fullsized":"2023-10-16T08:39:25.956207+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:30.649639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945004+0000","last_change":"2023-10-16T08:38:45.639658+0000","last_active":"2023-10-16T08:39:25.945004+0000","last_peered":"2023-10-16T08:39:25.945004+0000","last_clean":"2023-10-16T08:39:25.945004+0000","last_became_active":"2023-10-16T08:38:45.638702+0000","last_became_peered":"2023-10-16T08:38:45.638702+0000","last_unstale":"2023-10-16T08:39:25.945004+0000","last_undegraded":"2023-10-16T08:39:25.945004+0000","last_fullsized":"2023-10-16T08:39:25.945004+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'3","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.008178+0000","last_change":"2023-10-16T08:38:44.648715+0000","last_active":"2023-10-16T08:39:27.008178+0000","last_peered":"2023-10-16T08:39:27.008178+0000","last_clean":"2023-10-16T08:39:27.008178+0000","last_became_active":"2023-10-16T08:38:44.648312+0000","last_became_peered":"2023-10-16T08:38:44.648312+0000","last_unstale":"2023-10-16T08:39:27.008178+0000","last_undegraded":"2023-10-16T08:39:27.008178+0000","last_fullsized":"2023-10-16T08:39:27.008178+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955768+0000","last_change":"2023-10-16T08:38:45.663402+0000","last_active":"2023-10-16T08:39:25.955768+0000","last_peered":"2023-10-16T08:39:25.955768+0000","last_clean":"2023-10-16T08:39:25.955768+0000","last_became_active":"2023-10-16T08:38:45.662442+0000","last_became_peered":"2023-10-16T08:38:45.662442+0000","last_unstale":"2023-10-16T08:39:25.955768+0000","last_undegraded":"2023-10-16T08:39:25.955768+0000","last_fullsized":"2023-10-16T08:39:25.955768+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:44:29.665954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949981+0000","last_change":"2023-10-16T08:38:44.650015+0000","last_active":"2023-10-16T08:39:25.949981+0000","last_peered":"2023-10-16T08:39:25.949981+0000","last_clean":"2023-10-16T08:39:25.949981+0000","last_became_active":"2023-10-16T08:38:44.648155+0000","last_became_peered":"2023-10-16T08:38:44.648155+0000","last_unstale":"2023-10-16T08:39:25.949981+0000","last_undegraded":"2023-10-16T08:39:25.949981+0000","last_fullsized":"2023-10-16T08:39:25.949981+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402266+0000","last_change":"2023-10-16T08:38:45.637568+0000","last_active":"2023-10-16T08:39:26.402266+0000","last_peered":"2023-10-16T08:39:26.402266+0000","last_clean":"2023-10-16T08:39:26.402266+0000","last_became_active":"2023-10-16T08:38:45.636751+0000","last_became_peered":"2023-10-16T08:38:45.636751+0000","last_unstale":"2023-10-16T08:39:26.402266+0000","last_undegraded":"2023-10-16T08:39:26.402266+0000","last_fullsized":"2023-10-16T08:39:26.402266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.954444+0000","last_change":"2023-10-16T08:38:44.649257+0000","last_active":"2023-10-16T08:39:25.954444+0000","last_peered":"2023-10-16T08:39:25.954444+0000","last_clean":"2023-10-16T08:39:25.954444+0000","last_became_active":"2023-10-16T08:38:44.646990+0000","last_became_peered":"2023-10-16T08:38:44.646990+0000","last_unstale":"2023-10-16T08:39:25.954444+0000","last_undegraded":"2023-10-16T08:39:25.954444+0000","last_fullsized":"2023-10-16T08:39:25.954444+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:56:05.359655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.403442+0000","last_change":"2023-10-16T08:38:45.648228+0000","last_active":"2023-10-16T08:39:26.403442+0000","last_peered":"2023-10-16T08:39:26.403442+0000","last_clean":"2023-10-16T08:39:26.403442+0000","last_became_active":"2023-10-16T08:38:45.647449+0000","last_became_peered":"2023-10-16T08:38:45.647449+0000","last_unstale":"2023-10-16T08:39:26.403442+0000","last_undegraded":"2023-10-16T08:39:26.403442+0000","last_fullsized":"2023-10-16T08:39:26.403442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:20.779617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.944878+0000","last_change":"2023-10-16T08:38:44.631354+0000","last_active":"2023-10-16T08:39:25.944878+0000","last_peered":"2023-10-16T08:39:25.944878+0000","last_clean":"2023-10-16T08:39:25.944878+0000","last_became_active":"2023-10-16T08:38:44.630451+0000","last_became_peered":"2023-10-16T08:38:44.630451+0000","last_unstale":"2023-10-16T08:39:25.944878+0000","last_undegraded":"2023-10-16T08:39:25.944878+0000","last_fullsized":"2023-10-16T08:39:25.944878+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:10.980982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955719+0000","last_change":"2023-10-16T08:38:45.663898+0000","last_active":"2023-10-16T08:39:25.955719+0000","last_peered":"2023-10-16T08:39:25.955719+0000","last_clean":"2023-10-16T08:39:25.955719+0000","last_became_active":"2023-10-16T08:38:45.663100+0000","last_became_peered":"2023-10-16T08:38:45.663100+0000","last_unstale":"2023-10-16T08:39:25.955719+0000","last_undegraded":"2023-10-16T08:39:25.955719+0000","last_fullsized":"2023-10-16T08:39:25.955719+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:03:49.716770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.945128+0000","last_change":"2023-10-16T08:38:44.646987+0000","last_active":"2023-10-16T08:39:25.945128+0000","last_peered":"2023-10-16T08:39:25.945128+0000","last_clean":"2023-10-16T08:39:25.945128+0000","last_became_active":"2023-10-16T08:38:44.646580+0000","last_became_peered":"2023-10-16T08:38:44.646580+0000","last_unstale":"2023-10-16T08:39:25.945128+0000","last_undegraded":"2023-10-16T08:39:25.945128+0000","last_fullsized":"2023-10-16T08:39:25.945128+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956260+0000","last_change":"2023-10-16T08:38:45.662894+0000","last_active":"2023-10-16T08:39:25.956260+0000","last_peered":"2023-10-16T08:39:25.956260+0000","last_clean":"2023-10-16T08:39:25.956260+0000","last_became_active":"2023-10-16T08:38:45.662226+0000","last_became_peered":"2023-10-16T08:38:45.662226+0000","last_unstale":"2023-10-16T08:39:25.956260+0000","last_undegraded":"2023-10-16T08:39:25.956260+0000","last_fullsized":"2023-10-16T08:39:25.956260+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:30:31.660494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955552+0000","last_change":"2023-10-16T08:38:44.660571+0000","last_active":"2023-10-16T08:39:25.955552+0000","last_peered":"2023-10-16T08:39:25.955552+0000","last_clean":"2023-10-16T08:39:25.955552+0000","last_became_active":"2023-10-16T08:38:44.659976+0000","last_became_peered":"2023-10-16T08:38:44.659976+0000","last_unstale":"2023-10-16T08:39:25.955552+0000","last_undegraded":"2023-10-16T08:39:25.955552+0000","last_fullsized":"2023-10-16T08:39:25.955552+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:10.695583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.949392+0000","last_change":"2023-10-16T08:38:45.665200+0000","last_active":"2023-10-16T08:39:25.949392+0000","last_peered":"2023-10-16T08:39:25.949392+0000","last_clean":"2023-10-16T08:39:25.949392+0000","last_became_active":"2023-10-16T08:38:45.664616+0000","last_became_peered":"2023-10-16T08:38:45.664616+0000","last_unstale":"2023-10-16T08:39:25.949392+0000","last_undegraded":"2023-10-16T08:39:25.949392+0000","last_fullsized":"2023-10-16T08:39:25.949392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:15:01.212579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.953546+0000","last_change":"2023-10-16T08:38:45.680892+0000","last_active":"2023-10-16T08:39:25.953546+0000","last_peered":"2023-10-16T08:39:25.953546+0000","last_clean":"2023-10-16T08:39:25.953546+0000","last_became_active":"2023-10-16T08:38:45.680231+0000","last_became_peered":"2023-10-16T08:38:45.680231+0000","last_unstale":"2023-10-16T08:39:25.953546+0000","last_undegraded":"2023-10-16T08:39:25.953546+0000","last_fullsized":"2023-10-16T08:39:25.953546+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:58:51.599709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.956015+0000","last_change":"2023-10-16T08:38:44.664269+0000","last_active":"2023-10-16T08:39:25.956015+0000","last_peered":"2023-10-16T08:39:25.956015+0000","last_clean":"2023-10-16T08:39:25.956015+0000","last_became_active":"2023-10-16T08:38:44.663674+0000","last_became_peered":"2023-10-16T08:38:44.663674+0000","last_unstale":"2023-10-16T08:39:25.956015+0000","last_undegraded":"2023-10-16T08:39:25.956015+0000","last_fullsized":"2023-10-16T08:39:25.956015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:39:28.788944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.951695+0000","last_change":"2023-10-16T08:38:45.680890+0000","last_active":"2023-10-16T08:39:25.951695+0000","last_peered":"2023-10-16T08:39:25.951695+0000","last_clean":"2023-10-16T08:39:25.951695+0000","last_became_active":"2023-10-16T08:38:45.680266+0000","last_became_peered":"2023-10-16T08:38:45.680266+0000","last_unstale":"2023-10-16T08:39:25.951695+0000","last_undegraded":"2023-10-16T08:39:25.951695+0000","last_fullsized":"2023-10-16T08:39:25.951695+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:12:26.292137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"19'2","reported_seq":16,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955778+0000","last_change":"2023-10-16T08:38:44.653447+0000","last_active":"2023-10-16T08:39:25.955778+0000","last_peered":"2023-10-16T08:39:25.955778+0000","last_clean":"2023-10-16T08:39:25.955778+0000","last_became_active":"2023-10-16T08:38:44.652658+0000","last_became_peered":"2023-10-16T08:38:44.652658+0000","last_unstale":"2023-10-16T08:39:25.955778+0000","last_undegraded":"2023-10-16T08:39:25.955778+0000","last_fullsized":"2023-10-16T08:39:25.955778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:41:22.350066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402179+0000","last_change":"2023-10-16T08:38:45.665041+0000","last_active":"2023-10-16T08:39:26.402179+0000","last_peered":"2023-10-16T08:39:26.402179+0000","last_clean":"2023-10-16T08:39:26.402179+0000","last_became_active":"2023-10-16T08:38:45.664274+0000","last_became_peered":"2023-10-16T08:38:45.664274+0000","last_unstale":"2023-10-16T08:39:26.402179+0000","last_undegraded":"2023-10-16T08:39:26.402179+0000","last_fullsized":"2023-10-16T08:39:26.402179+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:24:11.706438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":15,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955588+0000","last_change":"2023-10-16T08:38:44.645754+0000","last_active":"2023-10-16T08:39:25.955588+0000","last_peered":"2023-10-16T08:39:25.955588+0000","last_clean":"2023-10-16T08:39:25.955588+0000","last_became_active":"2023-10-16T08:38:44.645000+0000","last_became_peered":"2023-10-16T08:38:44.645000+0000","last_unstale":"2023-10-16T08:39:25.955588+0000","last_undegraded":"2023-10-16T08:39:25.955588+0000","last_fullsized":"2023-10-16T08:39:25.955588+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:54:28.528430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":13,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:25.955495+0000","last_change":"2023-10-16T08:38:45.664252+0000","last_active":"2023-10-16T08:39:25.955495+0000","last_peered":"2023-10-16T08:39:25.955495+0000","last_clean":"2023-10-16T08:39:25.955495+0000","last_became_active":"2023-10-16T08:38:45.663538+0000","last_became_peered":"2023-10-16T08:38:45.663538+0000","last_unstale":"2023-10-16T08:39:25.955495+0000","last_undegraded":"2023-10-16T08:39:25.955495+0000","last_fullsized":"2023-10-16T08:39:25.955495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:46:20.744103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"19'4","reported_seq":18,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.402379+0000","last_change":"2023-10-16T08:38:44.646321+0000","last_active":"2023-10-16T08:39:26.402379+0000","last_peered":"2023-10-16T08:39:26.402379+0000","last_clean":"2023-10-16T08:39:26.402379+0000","last_became_active":"2023-10-16T08:38:44.645854+0000","last_became_peered":"2023-10-16T08:38:44.645854+0000","last_unstale":"2023-10-16T08:39:26.402379+0000","last_undegraded":"2023-10-16T08:39:26.402379+0000","last_fullsized":"2023-10-16T08:39:26.402379+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:55:10.980982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6065,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":12130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":53,"ondisk_log_size":53,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":827392,"data_stored":674262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.93300000000000005}]},{"osd":1,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.80800000000000005}]},{"osd":2,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.89100000000000001},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.876}]},{"osd":3,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.83399999999999996}]},{"osd":4,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.58599999999999997},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.754}]},{"osd":5,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":0.95999999999999996}]},{"osd":6,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.71099999999999997},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.64800000000000002}]}]},{"osd":6,"up_from":13,"seq":55834574865,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":88579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.153},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.0840000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.70299999999999996},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":1.113}]},{"osd":2,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.84599999999999997}]},{"osd":3,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":1.198},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":1.016}]},{"osd":4,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":1.4299999999999999}]},{"osd":5,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.90000000000000002},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.3560000000000001}]},{"osd":7,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.04,"5min":2.04,"15min":2.04},"last":1.2789999999999999},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.0509999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":233472,"data_stored":83402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":1.0309999999999999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.0580000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":1.101}]},{"osd":2,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.88800000000000001}]},{"osd":3,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.98999999999999999},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":0.73999999999999999}]},{"osd":4,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":1.079},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":1.1499999999999999}]},{"osd":6,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.2689999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.1779999999999999}]},{"osd":7,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":1.232},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.198}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27260,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344580,"statfs":{"total":96636764160,"available":96608849920,"internally_reserved":0,"allocated":241664,"data_stored":83438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":1.081},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.1419999999999999}]},{"osd":1,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.004}]},{"osd":2,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":1.212},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.75}]},{"osd":3,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.83499999999999996},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.96899999999999997}]},{"osd":5,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.877},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":1.046}]},{"osd":6,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":0.68200000000000005},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":1.181}]},{"osd":7,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":0.93799999999999994},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.153,"5min":2.153,"15min":2.153},"last":1.109}]}]},{"osd":3,"up_from":13,"seq":55834574866,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":88579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.84699999999999998}]},{"osd":1,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.64000000000000001}]},{"osd":2,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":1.069},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.74399999999999999}]},{"osd":4,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.97699999999999998},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":0.95999999999999996}]},{"osd":5,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":1.014}]},{"osd":6,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.2989999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.04}]},{"osd":7,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":0.93999999999999995},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.242}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":229376,"data_stored":83312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.76200000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.97599999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.70099999999999996}]},{"osd":4,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":1.0920000000000001}]},{"osd":5,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":1.2170000000000001},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":0.85699999999999998}]},{"osd":6,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.26},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":1.0509999999999999}]},{"osd":7,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":1.179},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.135}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":237568,"data_stored":83472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.78500000000000003}]},{"osd":2,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":1.069}]},{"osd":3,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.96699999999999997}]},{"osd":4,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.0369999999999999},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.85899999999999999}]},{"osd":5,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.008},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.175}]},{"osd":6,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.26},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.93799999999999994}]},{"osd":7,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.3420000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.5449999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":1.3140000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27840,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344000,"statfs":{"total":96636764160,"available":96608256000,"internally_reserved":0,"allocated":835584,"data_stored":674318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.57099999999999995}]},{"osd":2,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.1220000000000001}]},{"osd":3,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.629},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.95699999999999996}]},{"osd":4,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.1459999999999999}]},{"osd":5,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.85999999999999999},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":0.90700000000000003}]},{"osd":6,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.087},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.169}]},{"osd":7,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.03},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.99299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:39:48.727 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.727 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.727 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.728 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.728 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.729 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.729 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.729 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.730 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.730 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.730 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.731 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.731 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.732 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.732 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.732 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.733 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.733 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.733 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.734 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.734 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.735 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.735 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.737 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.738 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.738 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.738 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.739 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.739 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.740 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.740 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.740 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.741 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.741 INFO:tasks.ceph:pgid 2.1b last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.741 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.742 INFO:tasks.ceph:pgid 2.1a last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.742 INFO:tasks.ceph:pgid 2.19 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.743 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.743 INFO:tasks.ceph:pgid 2.18 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.743 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.744 INFO:tasks.ceph:pgid 2.17 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.744 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.744 INFO:tasks.ceph:pgid 2.16 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.745 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.745 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.746 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.746 INFO:tasks.ceph:pgid 2.14 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.746 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.747 INFO:tasks.ceph:pgid 2.13 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.747 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.747 INFO:tasks.ceph:pgid 2.12 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.748 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.748 INFO:tasks.ceph:pgid 2.11 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.749 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.749 INFO:tasks.ceph:pgid 2.10 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.749 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.750 INFO:tasks.ceph:pgid 2.f last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.750 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.750 INFO:tasks.ceph:pgid 2.e last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.751 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.751 INFO:tasks.ceph:pgid 2.d last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.752 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.752 INFO:tasks.ceph:pgid 2.c last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.752 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.753 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-16T08:38:33.264692+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=33, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.753 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.754 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.754 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.754 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.755 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.755 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.756 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.756 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.756 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.757 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.757 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.757 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.758 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.758 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.758 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.759 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.759 INFO:tasks.ceph:pgid 2.8 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.760 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.760 INFO:tasks.ceph:pgid 2.9 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.760 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.761 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.761 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.761 INFO:tasks.ceph:pgid 2.b last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.762 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.762 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.763 INFO:tasks.ceph:pgid 2.1c last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.763 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.763 INFO:tasks.ceph:pgid 2.1d last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.764 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.764 INFO:tasks.ceph:pgid 2.1e last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.764 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.765 INFO:tasks.ceph:pgid 2.1f last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:39:48.765 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-16T08:40:08.766 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:40:08.899 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.898+0000 7febeaae6640 1 Processor -- start 2023-10-16T08:40:08.900 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.898+0000 7febeaae6640 1 -- start start 2023-10-16T08:40:08.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.898+0000 7febeaae6640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7febe415cf40 0x7febe40ccef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:08.901 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.898+0000 7febeaae6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7febe40ade00 0x7febe40ac8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:08.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.898+0000 7febeaae6640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7febe406b540 con 0x7febe415d320 2023-10-16T08:40:08.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.898+0000 7febeaae6640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7febe406fbf0 con 0x7febe415cf40 2023-10-16T08:40:08.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.898+0000 7febeaae6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe406b7e0 con 0x7febe40ade00 2023-10-16T08:40:08.902 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.898+0000 7febe9083640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7febe40ade00 0x7febe40ac8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:08.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.898+0000 7febe8882640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7febe415cf40 0x7febe40ccef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:08.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.898+0000 7febe3fff640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7febe415d320 0x7febe406f4e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.40:57770/0 (socket says 172.21.15.40:57770) 2023-10-16T08:40:08.903 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.898+0000 7febe3fff640 1 -- 172.21.15.40:0/1125472082 learned_addr learned my addr 172.21.15.40:0/1125472082 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:40:08.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.898+0000 7febe9083640 1 -- 172.21.15.40:0/1125472082 >> v2:172.21.15.86:3300/0 conn(0x7febe415cf40 msgr2=0x7febe40ccef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:40:08.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.898+0000 7febe9083640 1 --2- 172.21.15.40:0/1125472082 >> v2:172.21.15.86:3300/0 conn(0x7febe415cf40 0x7febe40ccef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:08.906 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.898+0000 7febe9083640 1 -- 172.21.15.40:0/1125472082 >> v1:172.21.15.86:6789/0 conn(0x7febe415d320 legacy=0x7febe406f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:08.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.898+0000 7febe9083640 1 -- 172.21.15.40:0/1125472082 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febe406bac0 con 0x7febe40ade00 2023-10-16T08:40:08.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.898+0000 7febe8882640 1 --2- 172.21.15.40:0/1125472082 >> v2:172.21.15.86:3300/0 conn(0x7febe415cf40 0x7febe40ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:40:08.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.902+0000 7febe9083640 1 --2- 172.21.15.40:0/1125472082 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7febe40ade00 0x7febe40ac8e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7febdc0097c0 tx=0x7febdc01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6441f52b3b878e83 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:08.907 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.902+0000 7febe37fe640 1 -- 172.21.15.40:0/1125472082 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc0ad660 con 0x7febe40ade00 2023-10-16T08:40:08.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.902+0000 7febe37fe640 1 -- 172.21.15.40:0/1125472082 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febdc0ad7c0 con 0x7febe40ade00 2023-10-16T08:40:08.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.902+0000 7febe37fe640 1 -- 172.21.15.40:0/1125472082 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc0adb50 con 0x7febe40ade00 2023-10-16T08:40:08.908 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.902+0000 7febeaae6640 1 -- 172.21.15.40:0/1125472082 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7febe40ade00 msgr2=0x7febe40ac8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:08.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.902+0000 7febeaae6640 1 --2- 172.21.15.40:0/1125472082 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7febe40ade00 0x7febe40ac8e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7febdc0097c0 tx=0x7febdc01c450 comp rx=0 tx=0).stop 2023-10-16T08:40:08.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.902+0000 7febeaae6640 1 -- 172.21.15.40:0/1125472082 shutdown_connections 2023-10-16T08:40:08.909 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.902+0000 7febeaae6640 1 --2- 172.21.15.40:0/1125472082 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7febe40ade00 0x7febe40ac8e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:08.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.902+0000 7febeaae6640 1 --2- 172.21.15.40:0/1125472082 >> v2:172.21.15.86:3300/0 conn(0x7febe415cf40 0x7febe40ccef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:08.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.902+0000 7febeaae6640 1 -- 172.21.15.40:0/1125472082 >> 172.21.15.40:0/1125472082 conn(0x7febe40ce960 msgr2=0x7febe40ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:08.910 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.902+0000 7febeaae6640 1 -- 172.21.15.40:0/1125472082 shutdown_connections 2023-10-16T08:40:08.911 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.902+0000 7febeaae6640 1 -- 172.21.15.40:0/1125472082 wait complete. 2023-10-16T08:40:08.913 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.902+0000 7febeaae6640 1 Processor -- start 2023-10-16T08:40:08.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.902+0000 7febeaae6640 1 -- start start 2023-10-16T08:40:08.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.902+0000 7febeaae6640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7febe40ade00 0x7febe414b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:08.914 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.902+0000 7febeaae6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febe415cf40 0x7febe4158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:08.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febeaae6640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febe415d320 0x7febe4159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:08.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febeaae6640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe414b7d0 con 0x7febe415d320 2023-10-16T08:40:08.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febeaae6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7febe4158320 con 0x7febe415cf40 2023-10-16T08:40:08.915 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febeaae6640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7febe4158600 con 0x7febe40ade00 2023-10-16T08:40:08.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febe9083640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febe415d320 0x7febe4159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:08.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febe9083640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febe415d320 0x7febe4159400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:59108/0 (socket says 172.21.15.40:59108) 2023-10-16T08:40:08.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febe9083640 1 -- 172.21.15.40:0/3976898346 learned_addr learned my addr 172.21.15.40:0/3976898346 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:40:08.916 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febe8882640 1 --2- 172.21.15.40:0/3976898346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7febe40ade00 0x7febe414b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:08.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febe3fff640 1 --2- 172.21.15.40:0/3976898346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febe415cf40 0x7febe4158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:08.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febe9083640 1 -- 172.21.15.40:0/3976898346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7febe40ade00 msgr2=0x7febe414b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:08.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febe9083640 1 --2- 172.21.15.40:0/3976898346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7febe40ade00 0x7febe414b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:08.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febe9083640 1 -- 172.21.15.40:0/3976898346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febe415cf40 msgr2=0x7febe4158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:08.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febe9083640 1 --2- 172.21.15.40:0/3976898346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febe415cf40 0x7febe4158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:08.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febe9083640 1 -- 172.21.15.40:0/3976898346 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7febd4009420 con 0x7febe415d320 2023-10-16T08:40:08.918 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febe9083640 1 --2- 172.21.15.40:0/3976898346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febe415d320 0x7febe4159400 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7febdc01a080 tx=0x7febdc0191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:08.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febe17fa640 1 -- 172.21.15.40:0/3976898346 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc0b6610 con 0x7febe415d320 2023-10-16T08:40:08.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febe17fa640 1 -- 172.21.15.40:0/3976898346 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7febdc0b6770 con 0x7febe415d320 2023-10-16T08:40:08.919 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febeaae6640 1 -- 172.21.15.40:0/3976898346 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7febdc009420 con 0x7febe415d320 2023-10-16T08:40:08.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febe17fa640 1 -- 172.21.15.40:0/3976898346 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7febdc0b6ae0 con 0x7febe415d320 2023-10-16T08:40:08.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febeaae6640 1 -- 172.21.15.40:0/3976898346 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7febe4158bc0 con 0x7febe415d320 2023-10-16T08:40:08.920 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febeaae6640 1 -- 172.21.15.40:0/3976898346 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7febe404e9c0 con 0x7febe415d320 2023-10-16T08:40:08.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febe17fa640 1 -- 172.21.15.40:0/3976898346 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7febdc0b4040 con 0x7febe415d320 2023-10-16T08:40:08.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febe17fa640 1 --2- 172.21.15.40:0/3976898346 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7febd00664d0 0x7febd0068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:08.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febe17fa640 1 -- 172.21.15.40:0/3976898346 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7febdc11e5c0 con 0x7febe415d320 2023-10-16T08:40:08.921 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.906+0000 7febe8882640 1 --2- 172.21.15.40:0/3976898346 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7febd00664d0 0x7febd0068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:08.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.910+0000 7febe8882640 1 --2- 172.21.15.40:0/3976898346 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7febd00664d0 0x7febd0068990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7febd4009c80 tx=0x7febd4008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:08.922 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:08.910+0000 7febe17fa640 1 -- 172.21.15.40:0/3976898346 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7febdc0eba90 con 0x7febe415d320 2023-10-16T08:40:09.069 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:09.066+0000 7febeaae6640 1 -- 172.21.15.40:0/3976898346 --> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7febe406c360 con 0x7febd00664d0 2023-10-16T08:40:09.093 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:09.090+0000 7febe17fa640 1 -- 172.21.15.40:0/3976898346 <== mgr.4111 v2:172.21.15.86:6832/1188970899 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286361 (secure 0 0 0) 0x7febd40098d0 con 0x7febd00664d0 2023-10-16T08:40:09.096 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:40:09.098 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-16T08:40:09.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:09.094+0000 7febeaae6640 1 -- 172.21.15.40:0/3976898346 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7febd00664d0 msgr2=0x7febd0068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:09.099 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:09.094+0000 7febeaae6640 1 --2- 172.21.15.40:0/3976898346 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7febd00664d0 0x7febd0068990 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7febd4009c80 tx=0x7febd4008040 comp rx=0 tx=0).stop 2023-10-16T08:40:09.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:09.094+0000 7febeaae6640 1 -- 172.21.15.40:0/3976898346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febe415d320 msgr2=0x7febe4159400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:09.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:09.094+0000 7febeaae6640 1 --2- 172.21.15.40:0/3976898346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febe415d320 0x7febe4159400 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7febdc01a080 tx=0x7febdc0191f0 comp rx=0 tx=0).stop 2023-10-16T08:40:09.100 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:09.094+0000 7febeaae6640 1 -- 172.21.15.40:0/3976898346 shutdown_connections 2023-10-16T08:40:09.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:09.094+0000 7febeaae6640 1 --2- 172.21.15.40:0/3976898346 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7febd00664d0 0x7febd0068990 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:09.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:09.094+0000 7febeaae6640 1 --2- 172.21.15.40:0/3976898346 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7febe415d320 0x7febe4159400 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:09.101 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:09.094+0000 7febeaae6640 1 --2- 172.21.15.40:0/3976898346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7febe415cf40 0x7febe4158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:09.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:09.094+0000 7febeaae6640 1 --2- 172.21.15.40:0/3976898346 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7febe40ade00 0x7febe414b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:09.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:09.094+0000 7febeaae6640 1 -- 172.21.15.40:0/3976898346 >> 172.21.15.40:0/3976898346 conn(0x7febe40ce960 msgr2=0x7febe40ced40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:09.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:09.094+0000 7febeaae6640 1 -- 172.21.15.40:0/3976898346 shutdown_connections 2023-10-16T08:40:09.102 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:09.094+0000 7febeaae6640 1 -- 172.21.15.40:0/3976898346 wait complete. 2023-10-16T08:40:09.115 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-10-16T08:40:08.967398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596433,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":220,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":189,"ondisk_log_size":189,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219152,"kb_used_data":3024,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755568,"statfs":{"total":773094113280,"available":772869701632,"internally_reserved":0,"allocated":3096576,"data_stored":1859362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004119"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.987199+0000","last_change":"2023-10-16T08:39:26.963087+0000","last_active":"2023-10-16T08:39:27.987199+0000","last_peered":"2023-10-16T08:39:27.987199+0000","last_clean":"2023-10-16T08:39:27.987199+0000","last_became_active":"2023-10-16T08:39:26.962446+0000","last_became_peered":"2023-10-16T08:39:26.962446+0000","last_unstale":"2023-10-16T08:39:27.987199+0000","last_undegraded":"2023-10-16T08:39:27.987199+0000","last_fullsized":"2023-10-16T08:39:27.987199+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.007933+0000","last_change":"2023-10-16T08:38:45.657659+0000","last_active":"2023-10-16T08:39:27.007933+0000","last_peered":"2023-10-16T08:39:27.007933+0000","last_clean":"2023-10-16T08:39:27.007933+0000","last_became_active":"2023-10-16T08:38:45.656884+0000","last_became_peered":"2023-10-16T08:38:45.656884+0000","last_unstale":"2023-10-16T08:39:27.007933+0000","last_undegraded":"2023-10-16T08:39:27.007933+0000","last_fullsized":"2023-10-16T08:39:27.007933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:06.160364+0000","last_change":"2023-10-16T08:40:06.160545+0000","last_active":"2023-10-16T08:40:06.160364+0000","last_peered":"2023-10-16T08:40:06.160364+0000","last_clean":"2023-10-16T08:40:06.160364+0000","last_became_active":"2023-10-16T08:38:45.629297+0000","last_became_peered":"2023-10-16T08:38:45.629297+0000","last_unstale":"2023-10-16T08:40:06.160364+0000","last_undegraded":"2023-10-16T08:40:06.160364+0000","last_fullsized":"2023-10-16T08:40:06.160364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:06.160109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:06.160109+0000","last_clean_scrub_stamp":"2023-10-16T08:40:06.160109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:29.917626+0000","scrub_duration":0.003797534,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.964979+0000","last_change":"2023-10-16T08:38:45.675996+0000","last_active":"2023-10-16T08:39:26.964979+0000","last_peered":"2023-10-16T08:39:26.964979+0000","last_clean":"2023-10-16T08:39:26.964979+0000","last_became_active":"2023-10-16T08:38:45.675327+0000","last_became_peered":"2023-10-16T08:38:45.675327+0000","last_unstale":"2023-10-16T08:39:26.964979+0000","last_undegraded":"2023-10-16T08:39:26.964979+0000","last_fullsized":"2023-10-16T08:39:26.964979+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.010266+0000","last_change":"2023-10-16T08:38:45.653550+0000","last_active":"2023-10-16T08:39:27.010266+0000","last_peered":"2023-10-16T08:39:27.010266+0000","last_clean":"2023-10-16T08:39:27.010266+0000","last_became_active":"2023-10-16T08:38:45.652808+0000","last_became_peered":"2023-10-16T08:38:45.652808+0000","last_unstale":"2023-10-16T08:39:27.010266+0000","last_undegraded":"2023-10-16T08:39:27.010266+0000","last_fullsized":"2023-10-16T08:39:27.010266+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.964829+0000","last_change":"2023-10-16T08:38:45.675086+0000","last_active":"2023-10-16T08:39:26.964829+0000","last_peered":"2023-10-16T08:39:26.964829+0000","last_clean":"2023-10-16T08:39:26.964829+0000","last_became_active":"2023-10-16T08:38:45.671146+0000","last_became_peered":"2023-10-16T08:38:45.671146+0000","last_unstale":"2023-10-16T08:39:26.964829+0000","last_undegraded":"2023-10-16T08:39:26.964829+0000","last_fullsized":"2023-10-16T08:39:26.964829+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.009048+0000","last_change":"2023-10-16T08:38:45.652595+0000","last_active":"2023-10-16T08:39:27.009048+0000","last_peered":"2023-10-16T08:39:27.009048+0000","last_clean":"2023-10-16T08:39:27.009048+0000","last_became_active":"2023-10-16T08:38:45.651848+0000","last_became_peered":"2023-10-16T08:38:45.651848+0000","last_unstale":"2023-10-16T08:39:27.009048+0000","last_undegraded":"2023-10-16T08:39:27.009048+0000","last_fullsized":"2023-10-16T08:39:27.009048+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:06.819154+0000","last_change":"2023-10-16T08:40:06.819332+0000","last_active":"2023-10-16T08:40:06.819154+0000","last_peered":"2023-10-16T08:40:06.819154+0000","last_clean":"2023-10-16T08:40:06.819154+0000","last_became_active":"2023-10-16T08:38:45.664702+0000","last_became_peered":"2023-10-16T08:38:45.664702+0000","last_unstale":"2023-10-16T08:40:06.819154+0000","last_undegraded":"2023-10-16T08:40:06.819154+0000","last_fullsized":"2023-10-16T08:40:06.819154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:06.818902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:06.818902+0000","last_clean_scrub_stamp":"2023-10-16T08:40:06.818902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:00.868499+0000","scrub_duration":0.0035812309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.010233+0000","last_change":"2023-10-16T08:38:45.667037+0000","last_active":"2023-10-16T08:39:27.010233+0000","last_peered":"2023-10-16T08:39:27.010233+0000","last_clean":"2023-10-16T08:39:27.010233+0000","last_became_active":"2023-10-16T08:38:45.666474+0000","last_became_peered":"2023-10-16T08:38:45.666474+0000","last_unstale":"2023-10-16T08:39:27.010233+0000","last_undegraded":"2023-10-16T08:39:27.010233+0000","last_fullsized":"2023-10-16T08:39:27.010233+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.960693+0000","last_change":"2023-10-16T08:38:45.669859+0000","last_active":"2023-10-16T08:39:26.960693+0000","last_peered":"2023-10-16T08:39:26.960693+0000","last_clean":"2023-10-16T08:39:26.960693+0000","last_became_active":"2023-10-16T08:38:45.668278+0000","last_became_peered":"2023-10-16T08:38:45.668278+0000","last_unstale":"2023-10-16T08:39:26.960693+0000","last_undegraded":"2023-10-16T08:39:26.960693+0000","last_fullsized":"2023-10-16T08:39:26.960693+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.960223+0000","last_change":"2023-10-16T08:38:45.664567+0000","last_active":"2023-10-16T08:39:26.960223+0000","last_peered":"2023-10-16T08:39:26.960223+0000","last_clean":"2023-10-16T08:39:26.960223+0000","last_became_active":"2023-10-16T08:38:45.663813+0000","last_became_peered":"2023-10-16T08:38:45.663813+0000","last_unstale":"2023-10-16T08:39:26.960223+0000","last_undegraded":"2023-10-16T08:39:26.960223+0000","last_fullsized":"2023-10-16T08:39:26.960223+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:58.863583+0000","last_change":"2023-10-16T08:39:58.863718+0000","last_active":"2023-10-16T08:39:58.863583+0000","last_peered":"2023-10-16T08:39:58.863583+0000","last_clean":"2023-10-16T08:39:58.863583+0000","last_became_active":"2023-10-16T08:38:45.675320+0000","last_became_peered":"2023-10-16T08:38:45.675320+0000","last_unstale":"2023-10-16T08:39:58.863583+0000","last_undegraded":"2023-10-16T08:39:58.863583+0000","last_fullsized":"2023-10-16T08:39:58.863583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:58.863355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:58.863355+0000","last_clean_scrub_stamp":"2023-10-16T08:39:58.863355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:17:12.041514+0000","scrub_duration":0.0033670039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.007723+0000","last_change":"2023-10-16T08:38:45.668056+0000","last_active":"2023-10-16T08:39:27.007723+0000","last_peered":"2023-10-16T08:39:27.007723+0000","last_clean":"2023-10-16T08:39:27.007723+0000","last_became_active":"2023-10-16T08:38:45.667428+0000","last_became_peered":"2023-10-16T08:38:45.667428+0000","last_unstale":"2023-10-16T08:39:27.007723+0000","last_undegraded":"2023-10-16T08:39:27.007723+0000","last_fullsized":"2023-10-16T08:39:27.007723+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:50.221350+0000","last_change":"2023-10-16T08:39:50.221441+0000","last_active":"2023-10-16T08:39:50.221350+0000","last_peered":"2023-10-16T08:39:50.221350+0000","last_clean":"2023-10-16T08:39:50.221350+0000","last_became_active":"2023-10-16T08:38:45.636785+0000","last_became_peered":"2023-10-16T08:38:45.636785+0000","last_unstale":"2023-10-16T08:39:50.221350+0000","last_undegraded":"2023-10-16T08:39:50.221350+0000","last_fullsized":"2023-10-16T08:39:50.221350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:50.221128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:50.221128+0000","last_clean_scrub_stamp":"2023-10-16T08:39:50.221128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:58:43.521146+0000","scrub_duration":0.002811554,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.011123+0000","last_change":"2023-10-16T08:38:45.656433+0000","last_active":"2023-10-16T08:39:27.011123+0000","last_peered":"2023-10-16T08:39:27.011123+0000","last_clean":"2023-10-16T08:39:27.011123+0000","last_became_active":"2023-10-16T08:38:45.655395+0000","last_became_peered":"2023-10-16T08:38:45.655395+0000","last_unstale":"2023-10-16T08:39:27.011123+0000","last_undegraded":"2023-10-16T08:39:27.011123+0000","last_fullsized":"2023-10-16T08:39:27.011123+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:04.505262+0000","last_change":"2023-10-16T08:40:04.505370+0000","last_active":"2023-10-16T08:40:04.505262+0000","last_peered":"2023-10-16T08:40:04.505262+0000","last_clean":"2023-10-16T08:40:04.505262+0000","last_became_active":"2023-10-16T08:38:45.663312+0000","last_became_peered":"2023-10-16T08:38:45.663312+0000","last_unstale":"2023-10-16T08:40:04.505262+0000","last_undegraded":"2023-10-16T08:40:04.505262+0000","last_fullsized":"2023-10-16T08:40:04.505262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:04.505050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:04.505050+0000","last_clean_scrub_stamp":"2023-10-16T08:40:04.505050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:15.343100+0000","scrub_duration":0.0035649169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.987175+0000","last_change":"2023-10-16T08:39:27.013093+0000","last_active":"2023-10-16T08:39:27.987175+0000","last_peered":"2023-10-16T08:39:27.987175+0000","last_clean":"2023-10-16T08:39:27.987175+0000","last_became_active":"2023-10-16T08:39:27.012068+0000","last_became_peered":"2023-10-16T08:39:27.012068+0000","last_unstale":"2023-10-16T08:39:27.987175+0000","last_undegraded":"2023-10-16T08:39:27.987175+0000","last_fullsized":"2023-10-16T08:39:27.987175+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:05.650748+0000","last_change":"2023-10-16T08:40:05.650848+0000","last_active":"2023-10-16T08:40:05.650748+0000","last_peered":"2023-10-16T08:40:05.650748+0000","last_clean":"2023-10-16T08:40:05.650748+0000","last_became_active":"2023-10-16T08:38:45.630641+0000","last_became_peered":"2023-10-16T08:38:45.630641+0000","last_unstale":"2023-10-16T08:40:05.650748+0000","last_undegraded":"2023-10-16T08:40:05.650748+0000","last_fullsized":"2023-10-16T08:40:05.650748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:05.650492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:05.650492+0000","last_clean_scrub_stamp":"2023-10-16T08:40:05.650492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:42:23.950253+0000","scrub_duration":0.0038292840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.010376+0000","last_change":"2023-10-16T08:38:45.650087+0000","last_active":"2023-10-16T08:39:27.010376+0000","last_peered":"2023-10-16T08:39:27.010376+0000","last_clean":"2023-10-16T08:39:27.010376+0000","last_became_active":"2023-10-16T08:38:45.649475+0000","last_became_peered":"2023-10-16T08:38:45.649475+0000","last_unstale":"2023-10-16T08:39:27.010376+0000","last_undegraded":"2023-10-16T08:39:27.010376+0000","last_fullsized":"2023-10-16T08:39:27.010376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.959707+0000","last_change":"2023-10-16T08:38:45.667336+0000","last_active":"2023-10-16T08:39:26.959707+0000","last_peered":"2023-10-16T08:39:26.959707+0000","last_clean":"2023-10-16T08:39:26.959707+0000","last_became_active":"2023-10-16T08:38:45.665291+0000","last_became_peered":"2023-10-16T08:38:45.665291+0000","last_unstale":"2023-10-16T08:39:26.959707+0000","last_undegraded":"2023-10-16T08:39:26.959707+0000","last_fullsized":"2023-10-16T08:39:26.959707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:04.141444+0000","last_change":"2023-10-16T08:40:04.141546+0000","last_active":"2023-10-16T08:40:04.141444+0000","last_peered":"2023-10-16T08:40:04.141444+0000","last_clean":"2023-10-16T08:40:04.141444+0000","last_became_active":"2023-10-16T08:38:45.652412+0000","last_became_peered":"2023-10-16T08:38:45.652412+0000","last_unstale":"2023-10-16T08:40:04.141444+0000","last_undegraded":"2023-10-16T08:40:04.141444+0000","last_fullsized":"2023-10-16T08:40:04.141444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:04.141224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:04.141224+0000","last_clean_scrub_stamp":"2023-10-16T08:40:04.141224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:07.519281+0000","scrub_duration":0.0035640720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:05.502168+0000","last_change":"2023-10-16T08:40:05.502348+0000","last_active":"2023-10-16T08:40:05.502168+0000","last_peered":"2023-10-16T08:40:05.502168+0000","last_clean":"2023-10-16T08:40:05.502168+0000","last_became_active":"2023-10-16T08:38:45.676844+0000","last_became_peered":"2023-10-16T08:38:45.676844+0000","last_unstale":"2023-10-16T08:40:05.502168+0000","last_undegraded":"2023-10-16T08:40:05.502168+0000","last_fullsized":"2023-10-16T08:40:05.502168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:05.501915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:05.501915+0000","last_clean_scrub_stamp":"2023-10-16T08:40:05.501915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:46:30.176743+0000","scrub_duration":0.0034663509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:05.864255+0000","last_change":"2023-10-16T08:40:05.864353+0000","last_active":"2023-10-16T08:40:05.864255+0000","last_peered":"2023-10-16T08:40:05.864255+0000","last_clean":"2023-10-16T08:40:05.864255+0000","last_became_active":"2023-10-16T08:38:45.662944+0000","last_became_peered":"2023-10-16T08:38:45.662944+0000","last_unstale":"2023-10-16T08:40:05.864255+0000","last_undegraded":"2023-10-16T08:40:05.864255+0000","last_fullsized":"2023-10-16T08:40:05.864255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:05.864043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:05.864043+0000","last_clean_scrub_stamp":"2023-10-16T08:40:05.864043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:39.715444+0000","scrub_duration":0.0038577170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:00.546243+0000","last_change":"2023-10-16T08:40:00.546291+0000","last_active":"2023-10-16T08:40:00.546243+0000","last_peered":"2023-10-16T08:40:00.546243+0000","last_clean":"2023-10-16T08:40:00.546243+0000","last_became_active":"2023-10-16T08:38:45.666641+0000","last_became_peered":"2023-10-16T08:38:45.666641+0000","last_unstale":"2023-10-16T08:40:00.546243+0000","last_undegraded":"2023-10-16T08:40:00.546243+0000","last_fullsized":"2023-10-16T08:40:00.546243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:00.546142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:00.546142+0000","last_clean_scrub_stamp":"2023-10-16T08:40:00.546142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:13:33.731843+0000","scrub_duration":0.0032222819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.008655+0000","last_change":"2023-10-16T08:38:45.670526+0000","last_active":"2023-10-16T08:39:27.008655+0000","last_peered":"2023-10-16T08:39:27.008655+0000","last_clean":"2023-10-16T08:39:27.008655+0000","last_became_active":"2023-10-16T08:38:45.669734+0000","last_became_peered":"2023-10-16T08:38:45.669734+0000","last_unstale":"2023-10-16T08:39:27.008655+0000","last_undegraded":"2023-10-16T08:39:27.008655+0000","last_fullsized":"2023-10-16T08:39:27.008655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:07.003998+0000","last_change":"2023-10-16T08:40:07.004096+0000","last_active":"2023-10-16T08:40:07.003998+0000","last_peered":"2023-10-16T08:40:07.003998+0000","last_clean":"2023-10-16T08:40:07.003998+0000","last_became_active":"2023-10-16T08:38:45.665972+0000","last_became_peered":"2023-10-16T08:38:45.665972+0000","last_unstale":"2023-10-16T08:40:07.003998+0000","last_undegraded":"2023-10-16T08:40:07.003998+0000","last_fullsized":"2023-10-16T08:40:07.003998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:07.003796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:07.003796+0000","last_clean_scrub_stamp":"2023-10-16T08:40:07.003796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:01.053394+0000","scrub_duration":0.0034323180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.009996+0000","last_change":"2023-10-16T08:38:45.667047+0000","last_active":"2023-10-16T08:39:27.009996+0000","last_peered":"2023-10-16T08:39:27.009996+0000","last_clean":"2023-10-16T08:39:27.009996+0000","last_became_active":"2023-10-16T08:38:45.666403+0000","last_became_peered":"2023-10-16T08:38:45.666403+0000","last_unstale":"2023-10-16T08:39:27.009996+0000","last_undegraded":"2023-10-16T08:39:27.009996+0000","last_fullsized":"2023-10-16T08:39:27.009996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:04.840228+0000","last_change":"2023-10-16T08:40:04.840318+0000","last_active":"2023-10-16T08:40:04.840228+0000","last_peered":"2023-10-16T08:40:04.840228+0000","last_clean":"2023-10-16T08:40:04.840228+0000","last_became_active":"2023-10-16T08:38:45.664572+0000","last_became_peered":"2023-10-16T08:38:45.664572+0000","last_unstale":"2023-10-16T08:40:04.840228+0000","last_undegraded":"2023-10-16T08:40:04.840228+0000","last_fullsized":"2023-10-16T08:40:04.840228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:04.840012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:04.840012+0000","last_clean_scrub_stamp":"2023-10-16T08:40:04.840012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:55:44.359515+0000","scrub_duration":0.0038448359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.959639+0000","last_change":"2023-10-16T08:38:45.667378+0000","last_active":"2023-10-16T08:39:26.959639+0000","last_peered":"2023-10-16T08:39:26.959639+0000","last_clean":"2023-10-16T08:39:26.959639+0000","last_became_active":"2023-10-16T08:38:45.666547+0000","last_became_peered":"2023-10-16T08:38:45.666547+0000","last_unstale":"2023-10-16T08:39:26.959639+0000","last_undegraded":"2023-10-16T08:39:26.959639+0000","last_fullsized":"2023-10-16T08:39:26.959639+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.987100+0000","last_change":"2023-10-16T08:39:27.010902+0000","last_active":"2023-10-16T08:39:27.987100+0000","last_peered":"2023-10-16T08:39:27.987100+0000","last_clean":"2023-10-16T08:39:27.987100+0000","last_became_active":"2023-10-16T08:39:27.009983+0000","last_became_peered":"2023-10-16T08:39:27.009983+0000","last_unstale":"2023-10-16T08:39:27.987100+0000","last_undegraded":"2023-10-16T08:39:27.987100+0000","last_fullsized":"2023-10-16T08:39:27.987100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:58.646552+0000","last_change":"2023-10-16T08:39:58.646655+0000","last_active":"2023-10-16T08:39:58.646552+0000","last_peered":"2023-10-16T08:39:58.646552+0000","last_clean":"2023-10-16T08:39:58.646552+0000","last_became_active":"2023-10-16T08:39:27.013658+0000","last_became_peered":"2023-10-16T08:39:27.013658+0000","last_unstale":"2023-10-16T08:39:58.646552+0000","last_undegraded":"2023-10-16T08:39:58.646552+0000","last_fullsized":"2023-10-16T08:39:58.646552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:58.646339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:58.646339+0000","last_clean_scrub_stamp":"2023-10-16T08:39:58.646339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:32:08.951921+0000","scrub_duration":0.0036955680000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:02.119406+0000","last_change":"2023-10-16T08:40:02.119578+0000","last_active":"2023-10-16T08:40:02.119406+0000","last_peered":"2023-10-16T08:40:02.119406+0000","last_clean":"2023-10-16T08:40:02.119406+0000","last_became_active":"2023-10-16T08:38:45.636762+0000","last_became_peered":"2023-10-16T08:38:45.636762+0000","last_unstale":"2023-10-16T08:40:02.119406+0000","last_undegraded":"2023-10-16T08:40:02.119406+0000","last_fullsized":"2023-10-16T08:40:02.119406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:02.119151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:02.119151+0000","last_clean_scrub_stamp":"2023-10-16T08:40:02.119151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:23:40.725505+0000","scrub_duration":0.00382139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:52.785339+0000","last_change":"2023-10-16T08:39:52.785439+0000","last_active":"2023-10-16T08:39:52.785339+0000","last_peered":"2023-10-16T08:39:52.785339+0000","last_clean":"2023-10-16T08:39:52.785339+0000","last_became_active":"2023-10-16T08:38:45.658808+0000","last_became_peered":"2023-10-16T08:38:45.658808+0000","last_unstale":"2023-10-16T08:39:52.785339+0000","last_undegraded":"2023-10-16T08:39:52.785339+0000","last_fullsized":"2023-10-16T08:39:52.785339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:52.785111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:52.785111+0000","last_clean_scrub_stamp":"2023-10-16T08:39:52.785111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:48.998080+0000","scrub_duration":0.0036557069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:48.153116+0000","last_change":"2023-10-16T08:39:48.153181+0000","last_active":"2023-10-16T08:39:48.153116+0000","last_peered":"2023-10-16T08:39:48.153116+0000","last_clean":"2023-10-16T08:39:48.153116+0000","last_became_active":"2023-10-16T08:38:44.643614+0000","last_became_peered":"2023-10-16T08:38:44.643614+0000","last_unstale":"2023-10-16T08:39:48.153116+0000","last_undegraded":"2023-10-16T08:39:48.153116+0000","last_fullsized":"2023-10-16T08:39:48.153116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"19'1","last_scrub_stamp":"2023-10-16T08:39:48.153004+0000","last_deep_scrub":"19'1","last_deep_scrub_stamp":"2023-10-16T08:39:48.153004+0000","last_clean_scrub_stamp":"2023-10-16T08:39:48.153004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:20.737033+0000","scrub_duration":0.002884073,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.959854+0000","last_change":"2023-10-16T08:38:45.659854+0000","last_active":"2023-10-16T08:39:26.959854+0000","last_peered":"2023-10-16T08:39:26.959854+0000","last_clean":"2023-10-16T08:39:26.959854+0000","last_became_active":"2023-10-16T08:38:45.658409+0000","last_became_peered":"2023-10-16T08:38:45.658409+0000","last_unstale":"2023-10-16T08:39:26.959854+0000","last_undegraded":"2023-10-16T08:39:26.959854+0000","last_fullsized":"2023-10-16T08:39:26.959854+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:00.189167+0000","last_change":"2023-10-16T08:40:00.189264+0000","last_active":"2023-10-16T08:40:00.189167+0000","last_peered":"2023-10-16T08:40:00.189167+0000","last_clean":"2023-10-16T08:40:00.189167+0000","last_became_active":"2023-10-16T08:38:44.658257+0000","last_became_peered":"2023-10-16T08:38:44.658257+0000","last_unstale":"2023-10-16T08:40:00.189167+0000","last_undegraded":"2023-10-16T08:40:00.189167+0000","last_fullsized":"2023-10-16T08:40:00.189167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"19'1","last_scrub_stamp":"2023-10-16T08:40:00.188936+0000","last_deep_scrub":"19'1","last_deep_scrub_stamp":"2023-10-16T08:40:00.188936+0000","last_clean_scrub_stamp":"2023-10-16T08:40:00.188936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:03.566993+0000","scrub_duration":0.0036410499999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:51.092373+0000","last_change":"2023-10-16T08:39:51.092469+0000","last_active":"2023-10-16T08:39:51.092373+0000","last_peered":"2023-10-16T08:39:51.092373+0000","last_clean":"2023-10-16T08:39:51.092373+0000","last_became_active":"2023-10-16T08:38:44.645931+0000","last_became_peered":"2023-10-16T08:38:44.645931+0000","last_unstale":"2023-10-16T08:39:51.092373+0000","last_undegraded":"2023-10-16T08:39:51.092373+0000","last_fullsized":"2023-10-16T08:39:51.092373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:51.092105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:51.092105+0000","last_clean_scrub_stamp":"2023-10-16T08:39:51.092105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:32:01.397687+0000","scrub_duration":0.0039256869999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:01.915584+0000","last_change":"2023-10-16T08:40:01.915678+0000","last_active":"2023-10-16T08:40:01.915584+0000","last_peered":"2023-10-16T08:40:01.915584+0000","last_clean":"2023-10-16T08:40:01.915584+0000","last_became_active":"2023-10-16T08:38:45.662072+0000","last_became_peered":"2023-10-16T08:38:45.662072+0000","last_unstale":"2023-10-16T08:40:01.915584+0000","last_undegraded":"2023-10-16T08:40:01.915584+0000","last_fullsized":"2023-10-16T08:40:01.915584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:01.915369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:01.915369+0000","last_clean_scrub_stamp":"2023-10-16T08:40:01.915369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:49:56.364066+0000","scrub_duration":0.0034239230000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:50.531962+0000","last_change":"2023-10-16T08:39:50.532059+0000","last_active":"2023-10-16T08:39:50.531962+0000","last_peered":"2023-10-16T08:39:50.531962+0000","last_clean":"2023-10-16T08:39:50.531962+0000","last_became_active":"2023-10-16T08:38:44.657065+0000","last_became_peered":"2023-10-16T08:38:44.657065+0000","last_unstale":"2023-10-16T08:39:50.531962+0000","last_undegraded":"2023-10-16T08:39:50.531962+0000","last_fullsized":"2023-10-16T08:39:50.531962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:50.531739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:50.531739+0000","last_clean_scrub_stamp":"2023-10-16T08:39:50.531739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:49:44.980436+0000","scrub_duration":0.0034347560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.010905+0000","last_change":"2023-10-16T08:38:45.668048+0000","last_active":"2023-10-16T08:39:27.010905+0000","last_peered":"2023-10-16T08:39:27.010905+0000","last_clean":"2023-10-16T08:39:27.010905+0000","last_became_active":"2023-10-16T08:38:45.667413+0000","last_became_peered":"2023-10-16T08:38:45.667413+0000","last_unstale":"2023-10-16T08:39:27.010905+0000","last_undegraded":"2023-10-16T08:39:27.010905+0000","last_fullsized":"2023-10-16T08:39:27.010905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:53.126900+0000","last_change":"2023-10-16T08:39:53.127012+0000","last_active":"2023-10-16T08:39:53.126900+0000","last_peered":"2023-10-16T08:39:53.126900+0000","last_clean":"2023-10-16T08:39:53.126900+0000","last_became_active":"2023-10-16T08:38:44.662383+0000","last_became_peered":"2023-10-16T08:38:44.662383+0000","last_unstale":"2023-10-16T08:39:53.126900+0000","last_undegraded":"2023-10-16T08:39:53.126900+0000","last_fullsized":"2023-10-16T08:39:53.126900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:53.126641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:53.126641+0000","last_clean_scrub_stamp":"2023-10-16T08:39:53.126641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:32:03.432222+0000","scrub_duration":0.0040293320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.965528+0000","last_change":"2023-10-16T08:38:45.679271+0000","last_active":"2023-10-16T08:39:26.965528+0000","last_peered":"2023-10-16T08:39:26.965528+0000","last_clean":"2023-10-16T08:39:26.965528+0000","last_became_active":"2023-10-16T08:38:45.678050+0000","last_became_peered":"2023-10-16T08:38:45.678050+0000","last_unstale":"2023-10-16T08:39:26.965528+0000","last_undegraded":"2023-10-16T08:39:26.965528+0000","last_fullsized":"2023-10-16T08:39:26.965528+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:51.509756+0000","last_change":"2023-10-16T08:39:51.509855+0000","last_active":"2023-10-16T08:39:51.509756+0000","last_peered":"2023-10-16T08:39:51.509756+0000","last_clean":"2023-10-16T08:39:51.509756+0000","last_became_active":"2023-10-16T08:38:44.652618+0000","last_became_peered":"2023-10-16T08:38:44.652618+0000","last_unstale":"2023-10-16T08:39:51.509756+0000","last_undegraded":"2023-10-16T08:39:51.509756+0000","last_fullsized":"2023-10-16T08:39:51.509756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:51.509535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:51.509535+0000","last_clean_scrub_stamp":"2023-10-16T08:39:51.509535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:55:31.029038+0000","scrub_duration":0.0032865199999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.959627+0000","last_change":"2023-10-16T08:38:45.656603+0000","last_active":"2023-10-16T08:39:26.959627+0000","last_peered":"2023-10-16T08:39:26.959627+0000","last_clean":"2023-10-16T08:39:26.959627+0000","last_became_active":"2023-10-16T08:38:45.655213+0000","last_became_peered":"2023-10-16T08:38:45.655213+0000","last_unstale":"2023-10-16T08:39:26.959627+0000","last_undegraded":"2023-10-16T08:39:26.959627+0000","last_fullsized":"2023-10-16T08:39:26.959627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"19'3","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.010254+0000","last_change":"2023-10-16T08:38:44.651637+0000","last_active":"2023-10-16T08:39:27.010254+0000","last_peered":"2023-10-16T08:39:27.010254+0000","last_clean":"2023-10-16T08:39:27.010254+0000","last_became_active":"2023-10-16T08:38:44.651359+0000","last_became_peered":"2023-10-16T08:38:44.651359+0000","last_unstale":"2023-10-16T08:39:27.010254+0000","last_undegraded":"2023-10-16T08:39:27.010254+0000","last_fullsized":"2023-10-16T08:39:27.010254+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:54.792955+0000","last_change":"2023-10-16T08:39:54.793071+0000","last_active":"2023-10-16T08:39:54.792955+0000","last_peered":"2023-10-16T08:39:54.792955+0000","last_clean":"2023-10-16T08:39:54.792955+0000","last_became_active":"2023-10-16T08:38:45.660611+0000","last_became_peered":"2023-10-16T08:38:45.660611+0000","last_unstale":"2023-10-16T08:39:54.792955+0000","last_undegraded":"2023-10-16T08:39:54.792955+0000","last_fullsized":"2023-10-16T08:39:54.792955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:54.792721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:54.792721+0000","last_clean_scrub_stamp":"2023-10-16T08:39:54.792721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:27.376750+0000","scrub_duration":0.0038456089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'20","reported_seq":43,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:59.159576+0000","last_change":"2023-10-16T08:39:59.159729+0000","last_active":"2023-10-16T08:39:59.159576+0000","last_peered":"2023-10-16T08:39:59.159576+0000","last_clean":"2023-10-16T08:39:59.159576+0000","last_became_active":"2023-10-16T08:38:44.647001+0000","last_became_peered":"2023-10-16T08:38:44.647001+0000","last_unstale":"2023-10-16T08:39:59.159576+0000","last_undegraded":"2023-10-16T08:39:59.159576+0000","last_fullsized":"2023-10-16T08:39:59.159576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'20","last_scrub_stamp":"2023-10-16T08:39:59.159359+0000","last_deep_scrub":"22'20","last_deep_scrub_stamp":"2023-10-16T08:39:59.159359+0000","last_clean_scrub_stamp":"2023-10-16T08:39:59.159359+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:42:17.459120+0000","scrub_duration":0.0099116120000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5267,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.009751+0000","last_change":"2023-10-16T08:38:45.668629+0000","last_active":"2023-10-16T08:39:27.009751+0000","last_peered":"2023-10-16T08:39:27.009751+0000","last_clean":"2023-10-16T08:39:27.009751+0000","last_became_active":"2023-10-16T08:38:45.667334+0000","last_became_peered":"2023-10-16T08:38:45.667334+0000","last_unstale":"2023-10-16T08:39:27.009751+0000","last_undegraded":"2023-10-16T08:39:27.009751+0000","last_fullsized":"2023-10-16T08:39:27.009751+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'4","reported_seq":27,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:46.198227+0000","last_change":"2023-10-16T08:39:46.198310+0000","last_active":"2023-10-16T08:39:46.198227+0000","last_peered":"2023-10-16T08:39:46.198227+0000","last_clean":"2023-10-16T08:39:46.198227+0000","last_became_active":"2023-10-16T08:38:44.655469+0000","last_became_peered":"2023-10-16T08:38:44.655469+0000","last_unstale":"2023-10-16T08:39:46.198227+0000","last_undegraded":"2023-10-16T08:39:46.198227+0000","last_fullsized":"2023-10-16T08:39:46.198227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4","last_scrub_stamp":"2023-10-16T08:39:46.198102+0000","last_deep_scrub":"22'4","last_deep_scrub_stamp":"2023-10-16T08:39:46.198102+0000","last_clean_scrub_stamp":"2023-10-16T08:39:46.198102+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:19:00.034900+0000","scrub_duration":0.00664833,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.959062+0000","last_change":"2023-10-16T08:38:45.653688+0000","last_active":"2023-10-16T08:39:26.959062+0000","last_peered":"2023-10-16T08:39:26.959062+0000","last_clean":"2023-10-16T08:39:26.959062+0000","last_became_active":"2023-10-16T08:38:45.650563+0000","last_became_peered":"2023-10-16T08:38:45.650563+0000","last_unstale":"2023-10-16T08:39:26.959062+0000","last_undegraded":"2023-10-16T08:39:26.959062+0000","last_fullsized":"2023-10-16T08:39:26.959062+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:56.567995+0000","last_change":"2023-10-16T08:39:56.568099+0000","last_active":"2023-10-16T08:39:56.567995+0000","last_peered":"2023-10-16T08:39:56.567995+0000","last_clean":"2023-10-16T08:39:56.567995+0000","last_became_active":"2023-10-16T08:38:44.662110+0000","last_became_peered":"2023-10-16T08:38:44.662110+0000","last_unstale":"2023-10-16T08:39:56.567995+0000","last_undegraded":"2023-10-16T08:39:56.567995+0000","last_fullsized":"2023-10-16T08:39:56.567995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:56.567779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:56.567779+0000","last_clean_scrub_stamp":"2023-10-16T08:39:56.567779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:37:13.876495+0000","scrub_duration":0.0037362279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.010449+0000","last_change":"2023-10-16T08:38:45.662166+0000","last_active":"2023-10-16T08:39:27.010449+0000","last_peered":"2023-10-16T08:39:27.010449+0000","last_clean":"2023-10-16T08:39:27.010449+0000","last_became_active":"2023-10-16T08:38:45.661595+0000","last_became_peered":"2023-10-16T08:38:45.661595+0000","last_unstale":"2023-10-16T08:39:27.010449+0000","last_undegraded":"2023-10-16T08:39:27.010449+0000","last_fullsized":"2023-10-16T08:39:27.010449+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:50.191434+0000","last_change":"2023-10-16T08:39:50.191539+0000","last_active":"2023-10-16T08:39:50.191434+0000","last_peered":"2023-10-16T08:39:50.191434+0000","last_clean":"2023-10-16T08:39:50.191434+0000","last_became_active":"2023-10-16T08:38:44.657153+0000","last_became_peered":"2023-10-16T08:38:44.657153+0000","last_unstale":"2023-10-16T08:39:50.191434+0000","last_undegraded":"2023-10-16T08:39:50.191434+0000","last_fullsized":"2023-10-16T08:39:50.191434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:50.191165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:50.191165+0000","last_clean_scrub_stamp":"2023-10-16T08:39:50.191165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:31:15.292095+0000","scrub_duration":0.0038904819999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:04.049549+0000","last_change":"2023-10-16T08:40:04.049648+0000","last_active":"2023-10-16T08:40:04.049549+0000","last_peered":"2023-10-16T08:40:04.049549+0000","last_clean":"2023-10-16T08:40:04.049549+0000","last_became_active":"2023-10-16T08:38:45.635633+0000","last_became_peered":"2023-10-16T08:38:45.635633+0000","last_unstale":"2023-10-16T08:40:04.049549+0000","last_undegraded":"2023-10-16T08:40:04.049549+0000","last_fullsized":"2023-10-16T08:40:04.049549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:04.049337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:04.049337+0000","last_clean_scrub_stamp":"2023-10-16T08:40:04.049337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:16.401945+0000","scrub_duration":0.0038362539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'12","reported_seq":35,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:46.606020+0000","last_change":"2023-10-16T08:39:46.606097+0000","last_active":"2023-10-16T08:39:46.606020+0000","last_peered":"2023-10-16T08:39:46.606020+0000","last_clean":"2023-10-16T08:39:46.606020+0000","last_became_active":"2023-10-16T08:38:44.637640+0000","last_became_peered":"2023-10-16T08:38:44.637640+0000","last_unstale":"2023-10-16T08:39:46.606020+0000","last_undegraded":"2023-10-16T08:39:46.606020+0000","last_fullsized":"2023-10-16T08:39:46.606020+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'12","last_scrub_stamp":"2023-10-16T08:39:46.605891+0000","last_deep_scrub":"22'12","last_deep_scrub_stamp":"2023-10-16T08:39:46.605891+0000","last_clean_scrub_stamp":"2023-10-16T08:39:46.605891+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:02.027806+0000","scrub_duration":0.0068877740000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.009271+0000","last_change":"2023-10-16T08:38:45.657286+0000","last_active":"2023-10-16T08:39:27.009271+0000","last_peered":"2023-10-16T08:39:27.009271+0000","last_clean":"2023-10-16T08:39:27.009271+0000","last_became_active":"2023-10-16T08:38:45.656651+0000","last_became_peered":"2023-10-16T08:38:45.656651+0000","last_unstale":"2023-10-16T08:39:27.009271+0000","last_undegraded":"2023-10-16T08:39:27.009271+0000","last_fullsized":"2023-10-16T08:39:27.009271+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'2","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:47.765260+0000","last_change":"2023-10-16T08:39:47.765365+0000","last_active":"2023-10-16T08:39:47.765260+0000","last_peered":"2023-10-16T08:39:47.765260+0000","last_clean":"2023-10-16T08:39:47.765260+0000","last_became_active":"2023-10-16T08:38:44.647887+0000","last_became_peered":"2023-10-16T08:38:44.647887+0000","last_unstale":"2023-10-16T08:39:47.765260+0000","last_undegraded":"2023-10-16T08:39:47.765260+0000","last_fullsized":"2023-10-16T08:39:47.765260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'2","last_scrub_stamp":"2023-10-16T08:39:47.765049+0000","last_deep_scrub":"22'2","last_deep_scrub_stamp":"2023-10-16T08:39:47.765049+0000","last_clean_scrub_stamp":"2023-10-16T08:39:47.765049+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:03.642054+0000","scrub_duration":0.0054068919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:49.280907+0000","last_change":"2023-10-16T08:39:49.281008+0000","last_active":"2023-10-16T08:39:49.280907+0000","last_peered":"2023-10-16T08:39:49.280907+0000","last_clean":"2023-10-16T08:39:49.280907+0000","last_became_active":"2023-10-16T08:38:45.656750+0000","last_became_peered":"2023-10-16T08:38:45.656750+0000","last_unstale":"2023-10-16T08:39:49.280907+0000","last_undegraded":"2023-10-16T08:39:49.280907+0000","last_fullsized":"2023-10-16T08:39:49.280907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:49.280644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:49.280644+0000","last_clean_scrub_stamp":"2023-10-16T08:39:49.280644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:51:41.348858+0000","scrub_duration":0.0036633019999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'3","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:56.656761+0000","last_change":"2023-10-16T08:39:56.656883+0000","last_active":"2023-10-16T08:39:56.656761+0000","last_peered":"2023-10-16T08:39:56.656761+0000","last_clean":"2023-10-16T08:39:56.656761+0000","last_became_active":"2023-10-16T08:38:44.646042+0000","last_became_peered":"2023-10-16T08:38:44.646042+0000","last_unstale":"2023-10-16T08:39:56.656761+0000","last_undegraded":"2023-10-16T08:39:56.656761+0000","last_fullsized":"2023-10-16T08:39:56.656761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2023-10-16T08:39:56.656531+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2023-10-16T08:39:56.656531+0000","last_clean_scrub_stamp":"2023-10-16T08:39:56.656531+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:31:21.757461+0000","scrub_duration":0.0076556489999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.010439+0000","last_change":"2023-10-16T08:38:45.670969+0000","last_active":"2023-10-16T08:39:27.010439+0000","last_peered":"2023-10-16T08:39:27.010439+0000","last_clean":"2023-10-16T08:39:27.010439+0000","last_became_active":"2023-10-16T08:38:45.670415+0000","last_became_peered":"2023-10-16T08:38:45.670415+0000","last_unstale":"2023-10-16T08:39:27.010439+0000","last_undegraded":"2023-10-16T08:39:27.010439+0000","last_fullsized":"2023-10-16T08:39:27.010439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'2","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:01.687399+0000","last_change":"2023-10-16T08:40:01.687511+0000","last_active":"2023-10-16T08:40:01.687399+0000","last_peered":"2023-10-16T08:40:01.687399+0000","last_clean":"2023-10-16T08:40:01.687399+0000","last_became_active":"2023-10-16T08:38:44.655260+0000","last_became_peered":"2023-10-16T08:38:44.655260+0000","last_unstale":"2023-10-16T08:40:01.687399+0000","last_undegraded":"2023-10-16T08:40:01.687399+0000","last_fullsized":"2023-10-16T08:40:01.687399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'2","last_scrub_stamp":"2023-10-16T08:40:01.687151+0000","last_deep_scrub":"22'2","last_deep_scrub_stamp":"2023-10-16T08:40:01.687151+0000","last_clean_scrub_stamp":"2023-10-16T08:40:01.687151+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:32:11.992732+0000","scrub_duration":0.0081170519999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:06.489937+0000","last_change":"2023-10-16T08:40:06.490039+0000","last_active":"2023-10-16T08:40:06.489937+0000","last_peered":"2023-10-16T08:40:06.489937+0000","last_clean":"2023-10-16T08:40:06.489937+0000","last_became_active":"2023-10-16T08:38:45.667627+0000","last_became_peered":"2023-10-16T08:38:45.667627+0000","last_unstale":"2023-10-16T08:40:06.489937+0000","last_undegraded":"2023-10-16T08:40:06.489937+0000","last_fullsized":"2023-10-16T08:40:06.489937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:06.489710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:06.489710+0000","last_clean_scrub_stamp":"2023-10-16T08:40:06.489710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:10:38.568870+0000","scrub_duration":0.003795746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:51.623054+0000","last_change":"2023-10-16T08:39:51.623164+0000","last_active":"2023-10-16T08:39:51.623054+0000","last_peered":"2023-10-16T08:39:51.623054+0000","last_clean":"2023-10-16T08:39:51.623054+0000","last_became_active":"2023-10-16T08:38:44.661736+0000","last_became_peered":"2023-10-16T08:38:44.661736+0000","last_unstale":"2023-10-16T08:39:51.623054+0000","last_undegraded":"2023-10-16T08:39:51.623054+0000","last_fullsized":"2023-10-16T08:39:51.623054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:51.622816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:51.622816+0000","last_clean_scrub_stamp":"2023-10-16T08:39:51.622816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:29.535131+0000","scrub_duration":0.0038193530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.009958+0000","last_change":"2023-10-16T08:38:45.667423+0000","last_active":"2023-10-16T08:39:27.009958+0000","last_peered":"2023-10-16T08:39:27.009958+0000","last_clean":"2023-10-16T08:39:27.009958+0000","last_became_active":"2023-10-16T08:38:45.666390+0000","last_became_peered":"2023-10-16T08:38:45.666390+0000","last_unstale":"2023-10-16T08:39:27.009958+0000","last_undegraded":"2023-10-16T08:39:27.009958+0000","last_fullsized":"2023-10-16T08:39:27.009958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'119","reported_seq":178,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:50.308532+0000","last_change":"2023-10-16T08:39:50.308644+0000","last_active":"2023-10-16T08:39:50.308532+0000","last_peered":"2023-10-16T08:39:50.308532+0000","last_clean":"2023-10-16T08:39:50.308532+0000","last_became_active":"2023-10-16T08:38:34.310944+0000","last_became_peered":"2023-10-16T08:38:34.310944+0000","last_unstale":"2023-10-16T08:39:50.308532+0000","last_undegraded":"2023-10-16T08:39:50.308532+0000","last_fullsized":"2023-10-16T08:39:50.308532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'119","last_scrub_stamp":"2023-10-16T08:39:50.308327+0000","last_deep_scrub":"22'119","last_deep_scrub_stamp":"2023-10-16T08:39:50.308327+0000","last_clean_scrub_stamp":"2023-10-16T08:39:50.308327+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:14.065843+0000","scrub_duration":0.014557508,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:48.490807+0000","last_change":"2023-10-16T08:39:48.490904+0000","last_active":"2023-10-16T08:39:48.490807+0000","last_peered":"2023-10-16T08:39:48.490807+0000","last_clean":"2023-10-16T08:39:48.490807+0000","last_became_active":"2023-10-16T08:38:44.654277+0000","last_became_peered":"2023-10-16T08:38:44.654277+0000","last_unstale":"2023-10-16T08:39:48.490807+0000","last_undegraded":"2023-10-16T08:39:48.490807+0000","last_fullsized":"2023-10-16T08:39:48.490807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:48.490591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:48.490591+0000","last_clean_scrub_stamp":"2023-10-16T08:39:48.490591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:51:40.558806+0000","scrub_duration":0.0031485229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.007878+0000","last_change":"2023-10-16T08:38:45.679868+0000","last_active":"2023-10-16T08:39:27.007878+0000","last_peered":"2023-10-16T08:39:27.007878+0000","last_clean":"2023-10-16T08:39:27.007878+0000","last_became_active":"2023-10-16T08:38:45.679409+0000","last_became_peered":"2023-10-16T08:38:45.679409+0000","last_unstale":"2023-10-16T08:39:27.007878+0000","last_undegraded":"2023-10-16T08:39:27.007878+0000","last_fullsized":"2023-10-16T08:39:27.007878+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:55.336465+0000","last_change":"2023-10-16T08:39:55.336630+0000","last_active":"2023-10-16T08:39:55.336465+0000","last_peered":"2023-10-16T08:39:55.336465+0000","last_clean":"2023-10-16T08:39:55.336465+0000","last_became_active":"2023-10-16T08:38:44.651534+0000","last_became_peered":"2023-10-16T08:38:44.651534+0000","last_unstale":"2023-10-16T08:39:55.336465+0000","last_undegraded":"2023-10-16T08:39:55.336465+0000","last_fullsized":"2023-10-16T08:39:55.336465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:55.336229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:55.336229+0000","last_clean_scrub_stamp":"2023-10-16T08:39:55.336229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:37:12.644946+0000","scrub_duration":0.0081414639999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:52.170208+0000","last_change":"2023-10-16T08:39:52.170378+0000","last_active":"2023-10-16T08:39:52.170208+0000","last_peered":"2023-10-16T08:39:52.170208+0000","last_clean":"2023-10-16T08:39:52.170208+0000","last_became_active":"2023-10-16T08:38:45.650715+0000","last_became_peered":"2023-10-16T08:38:45.650715+0000","last_unstale":"2023-10-16T08:39:52.170208+0000","last_undegraded":"2023-10-16T08:39:52.170208+0000","last_fullsized":"2023-10-16T08:39:52.170208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:52.169935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:52.169935+0000","last_clean_scrub_stamp":"2023-10-16T08:39:52.169935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:48:51.865234+0000","scrub_duration":0.0041537630000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:53.623737+0000","last_change":"2023-10-16T08:39:53.623864+0000","last_active":"2023-10-16T08:39:53.623737+0000","last_peered":"2023-10-16T08:39:53.623737+0000","last_clean":"2023-10-16T08:39:53.623737+0000","last_became_active":"2023-10-16T08:38:44.645451+0000","last_became_peered":"2023-10-16T08:38:44.645451+0000","last_unstale":"2023-10-16T08:39:53.623737+0000","last_undegraded":"2023-10-16T08:39:53.623737+0000","last_fullsized":"2023-10-16T08:39:53.623737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:53.623513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:53.623513+0000","last_clean_scrub_stamp":"2023-10-16T08:39:53.623513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:58.932498+0000","scrub_duration":0.0035529649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.008066+0000","last_change":"2023-10-16T08:38:45.641499+0000","last_active":"2023-10-16T08:39:27.008066+0000","last_peered":"2023-10-16T08:39:27.008066+0000","last_clean":"2023-10-16T08:39:27.008066+0000","last_became_active":"2023-10-16T08:38:45.640765+0000","last_became_peered":"2023-10-16T08:38:45.640765+0000","last_unstale":"2023-10-16T08:39:27.008066+0000","last_undegraded":"2023-10-16T08:39:27.008066+0000","last_fullsized":"2023-10-16T08:39:27.008066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:53.528625+0000","last_change":"2023-10-16T08:39:53.528786+0000","last_active":"2023-10-16T08:39:53.528625+0000","last_peered":"2023-10-16T08:39:53.528625+0000","last_clean":"2023-10-16T08:39:53.528625+0000","last_became_active":"2023-10-16T08:38:44.654622+0000","last_became_peered":"2023-10-16T08:38:44.654622+0000","last_unstale":"2023-10-16T08:39:53.528625+0000","last_undegraded":"2023-10-16T08:39:53.528625+0000","last_fullsized":"2023-10-16T08:39:53.528625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:53.528393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:53.528393+0000","last_clean_scrub_stamp":"2023-10-16T08:39:53.528393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:27.379794+0000","scrub_duration":0.0041697289999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:53.788332+0000","last_change":"2023-10-16T08:39:53.788477+0000","last_active":"2023-10-16T08:39:53.788332+0000","last_peered":"2023-10-16T08:39:53.788332+0000","last_clean":"2023-10-16T08:39:53.788332+0000","last_became_active":"2023-10-16T08:38:45.658594+0000","last_became_peered":"2023-10-16T08:38:45.658594+0000","last_unstale":"2023-10-16T08:39:53.788332+0000","last_undegraded":"2023-10-16T08:39:53.788332+0000","last_fullsized":"2023-10-16T08:39:53.788332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:53.788113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:53.788113+0000","last_clean_scrub_stamp":"2023-10-16T08:39:53.788113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:02.143993+0000","scrub_duration":0.0036952420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:45.752930+0000","last_change":"2023-10-16T08:39:45.752966+0000","last_active":"2023-10-16T08:39:45.752930+0000","last_peered":"2023-10-16T08:39:45.752930+0000","last_clean":"2023-10-16T08:39:45.752930+0000","last_became_active":"2023-10-16T08:38:44.633854+0000","last_became_peered":"2023-10-16T08:38:44.633854+0000","last_unstale":"2023-10-16T08:39:45.752930+0000","last_undegraded":"2023-10-16T08:39:45.752930+0000","last_fullsized":"2023-10-16T08:39:45.752930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:45.752829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:45.752829+0000","last_clean_scrub_stamp":"2023-10-16T08:39:45.752829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:18:59.589626+0000","scrub_duration":0.0015287009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":21,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.017458+0000","last_change":"2023-10-16T08:39:27.015974+0000","last_active":"2023-10-16T08:39:27.017458+0000","last_peered":"2023-10-16T08:39:27.017458+0000","last_clean":"2023-10-16T08:39:27.017458+0000","last_became_active":"2023-10-16T08:39:27.015266+0000","last_became_peered":"2023-10-16T08:39:27.015266+0000","last_unstale":"2023-10-16T08:39:27.017458+0000","last_undegraded":"2023-10-16T08:39:27.017458+0000","last_fullsized":"2023-10-16T08:39:27.017458+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:04.332824+0000","last_change":"2023-10-16T08:40:04.332989+0000","last_active":"2023-10-16T08:40:04.332824+0000","last_peered":"2023-10-16T08:40:04.332824+0000","last_clean":"2023-10-16T08:40:04.332824+0000","last_became_active":"2023-10-16T08:38:44.655001+0000","last_became_peered":"2023-10-16T08:38:44.655001+0000","last_unstale":"2023-10-16T08:40:04.332824+0000","last_undegraded":"2023-10-16T08:40:04.332824+0000","last_fullsized":"2023-10-16T08:40:04.332824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:04.332585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:04.332585+0000","last_clean_scrub_stamp":"2023-10-16T08:40:04.332585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:46:29.007413+0000","scrub_duration":0.003821592,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.008321+0000","last_change":"2023-10-16T08:38:45.639658+0000","last_active":"2023-10-16T08:39:27.008321+0000","last_peered":"2023-10-16T08:39:27.008321+0000","last_clean":"2023-10-16T08:39:27.008321+0000","last_became_active":"2023-10-16T08:38:45.638702+0000","last_became_peered":"2023-10-16T08:38:45.638702+0000","last_unstale":"2023-10-16T08:39:27.008321+0000","last_undegraded":"2023-10-16T08:39:27.008321+0000","last_fullsized":"2023-10-16T08:39:27.008321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'3","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:06.680541+0000","last_change":"2023-10-16T08:40:06.680647+0000","last_active":"2023-10-16T08:40:06.680541+0000","last_peered":"2023-10-16T08:40:06.680541+0000","last_clean":"2023-10-16T08:40:06.680541+0000","last_became_active":"2023-10-16T08:38:44.648312+0000","last_became_peered":"2023-10-16T08:38:44.648312+0000","last_unstale":"2023-10-16T08:40:06.680541+0000","last_undegraded":"2023-10-16T08:40:06.680541+0000","last_fullsized":"2023-10-16T08:40:06.680541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2023-10-16T08:40:06.680326+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2023-10-16T08:40:06.680326+0000","last_clean_scrub_stamp":"2023-10-16T08:40:06.680326+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:42:24.980087+0000","scrub_duration":0.0047609419999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.008881+0000","last_change":"2023-10-16T08:38:45.663402+0000","last_active":"2023-10-16T08:39:27.008881+0000","last_peered":"2023-10-16T08:39:27.008881+0000","last_clean":"2023-10-16T08:39:27.008881+0000","last_became_active":"2023-10-16T08:38:45.662442+0000","last_became_peered":"2023-10-16T08:38:45.662442+0000","last_unstale":"2023-10-16T08:39:27.008881+0000","last_undegraded":"2023-10-16T08:39:27.008881+0000","last_fullsized":"2023-10-16T08:39:27.008881+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'3","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:48.214304+0000","last_change":"2023-10-16T08:39:48.214423+0000","last_active":"2023-10-16T08:39:48.214304+0000","last_peered":"2023-10-16T08:39:48.214304+0000","last_clean":"2023-10-16T08:39:48.214304+0000","last_became_active":"2023-10-16T08:38:44.648155+0000","last_became_peered":"2023-10-16T08:38:44.648155+0000","last_unstale":"2023-10-16T08:39:48.214304+0000","last_undegraded":"2023-10-16T08:39:48.214304+0000","last_fullsized":"2023-10-16T08:39:48.214304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2023-10-16T08:39:48.214061+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2023-10-16T08:39:48.214061+0000","last_clean_scrub_stamp":"2023-10-16T08:39:48.214061+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:53.523046+0000","scrub_duration":0.0072707400000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.960519+0000","last_change":"2023-10-16T08:38:45.637568+0000","last_active":"2023-10-16T08:39:26.960519+0000","last_peered":"2023-10-16T08:39:26.960519+0000","last_clean":"2023-10-16T08:39:26.960519+0000","last_became_active":"2023-10-16T08:38:45.636751+0000","last_became_peered":"2023-10-16T08:38:45.636751+0000","last_unstale":"2023-10-16T08:39:26.960519+0000","last_undegraded":"2023-10-16T08:39:26.960519+0000","last_fullsized":"2023-10-16T08:39:26.960519+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:51.303765+0000","last_change":"2023-10-16T08:39:51.303866+0000","last_active":"2023-10-16T08:39:51.303765+0000","last_peered":"2023-10-16T08:39:51.303765+0000","last_clean":"2023-10-16T08:39:51.303765+0000","last_became_active":"2023-10-16T08:38:44.646990+0000","last_became_peered":"2023-10-16T08:38:44.646990+0000","last_unstale":"2023-10-16T08:39:51.303765+0000","last_undegraded":"2023-10-16T08:39:51.303765+0000","last_fullsized":"2023-10-16T08:39:51.303765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:51.303518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:51.303518+0000","last_clean_scrub_stamp":"2023-10-16T08:39:51.303518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:55:30.823021+0000","scrub_duration":0.0039969649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:48.807872+0000","last_change":"2023-10-16T08:39:48.807982+0000","last_active":"2023-10-16T08:39:48.807872+0000","last_peered":"2023-10-16T08:39:48.807872+0000","last_clean":"2023-10-16T08:39:48.807872+0000","last_became_active":"2023-10-16T08:38:45.647449+0000","last_became_peered":"2023-10-16T08:38:45.647449+0000","last_unstale":"2023-10-16T08:39:48.807872+0000","last_undegraded":"2023-10-16T08:39:48.807872+0000","last_fullsized":"2023-10-16T08:39:48.807872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:48.807616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:48.807616+0000","last_clean_scrub_stamp":"2023-10-16T08:39:48.807616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:04.229531+0000","scrub_duration":0.0038025490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:48.714882+0000","last_change":"2023-10-16T08:39:48.714951+0000","last_active":"2023-10-16T08:39:48.714882+0000","last_peered":"2023-10-16T08:39:48.714882+0000","last_clean":"2023-10-16T08:39:48.714882+0000","last_became_active":"2023-10-16T08:38:44.630451+0000","last_became_peered":"2023-10-16T08:38:44.630451+0000","last_unstale":"2023-10-16T08:39:48.714882+0000","last_undegraded":"2023-10-16T08:39:48.714882+0000","last_fullsized":"2023-10-16T08:39:48.714882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:48.714761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:48.714761+0000","last_clean_scrub_stamp":"2023-10-16T08:39:48.714761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:42:57.070641+0000","scrub_duration":0.0033819409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.007675+0000","last_change":"2023-10-16T08:38:45.663898+0000","last_active":"2023-10-16T08:39:27.007675+0000","last_peered":"2023-10-16T08:39:27.007675+0000","last_clean":"2023-10-16T08:39:27.007675+0000","last_became_active":"2023-10-16T08:38:45.663100+0000","last_became_peered":"2023-10-16T08:38:45.663100+0000","last_unstale":"2023-10-16T08:39:27.007675+0000","last_undegraded":"2023-10-16T08:39:27.007675+0000","last_fullsized":"2023-10-16T08:39:27.007675+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.009170+0000","last_change":"2023-10-16T08:38:44.646987+0000","last_active":"2023-10-16T08:39:27.009170+0000","last_peered":"2023-10-16T08:39:27.009170+0000","last_clean":"2023-10-16T08:39:27.009170+0000","last_became_active":"2023-10-16T08:38:44.646580+0000","last_became_peered":"2023-10-16T08:38:44.646580+0000","last_unstale":"2023-10-16T08:39:27.009170+0000","last_undegraded":"2023-10-16T08:39:27.009170+0000","last_fullsized":"2023-10-16T08:39:27.009170+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:43.584292+0000","last_clean_scrub_stamp":"2023-10-16T08:38:43.584292+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.007934+0000","last_change":"2023-10-16T08:38:45.662894+0000","last_active":"2023-10-16T08:39:27.007934+0000","last_peered":"2023-10-16T08:39:27.007934+0000","last_clean":"2023-10-16T08:39:27.007934+0000","last_became_active":"2023-10-16T08:38:45.662226+0000","last_became_peered":"2023-10-16T08:38:45.662226+0000","last_unstale":"2023-10-16T08:39:27.007934+0000","last_undegraded":"2023-10-16T08:39:27.007934+0000","last_fullsized":"2023-10-16T08:39:27.007934+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'3","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:53.387291+0000","last_change":"2023-10-16T08:39:53.387482+0000","last_active":"2023-10-16T08:39:53.387291+0000","last_peered":"2023-10-16T08:39:53.387291+0000","last_clean":"2023-10-16T08:39:53.387291+0000","last_became_active":"2023-10-16T08:38:44.659976+0000","last_became_peered":"2023-10-16T08:38:44.659976+0000","last_unstale":"2023-10-16T08:39:53.387291+0000","last_undegraded":"2023-10-16T08:39:53.387291+0000","last_fullsized":"2023-10-16T08:39:53.387291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2023-10-16T08:39:53.387049+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2023-10-16T08:39:53.387049+0000","last_clean_scrub_stamp":"2023-10-16T08:39:53.387049+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:27.238450+0000","scrub_duration":0.0071616939999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.959472+0000","last_change":"2023-10-16T08:38:45.665200+0000","last_active":"2023-10-16T08:39:26.959472+0000","last_peered":"2023-10-16T08:39:26.959472+0000","last_clean":"2023-10-16T08:39:26.959472+0000","last_became_active":"2023-10-16T08:38:45.664616+0000","last_became_peered":"2023-10-16T08:38:45.664616+0000","last_unstale":"2023-10-16T08:39:26.959472+0000","last_undegraded":"2023-10-16T08:39:26.959472+0000","last_fullsized":"2023-10-16T08:39:26.959472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.965356+0000","last_change":"2023-10-16T08:38:45.680892+0000","last_active":"2023-10-16T08:39:26.965356+0000","last_peered":"2023-10-16T08:39:26.965356+0000","last_clean":"2023-10-16T08:39:26.965356+0000","last_became_active":"2023-10-16T08:38:45.680231+0000","last_became_peered":"2023-10-16T08:38:45.680231+0000","last_unstale":"2023-10-16T08:39:26.965356+0000","last_undegraded":"2023-10-16T08:39:26.965356+0000","last_fullsized":"2023-10-16T08:39:26.965356+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'3","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:58.403119+0000","last_change":"2023-10-16T08:39:58.403247+0000","last_active":"2023-10-16T08:39:58.403119+0000","last_peered":"2023-10-16T08:39:58.403119+0000","last_clean":"2023-10-16T08:39:58.403119+0000","last_became_active":"2023-10-16T08:38:44.663674+0000","last_became_peered":"2023-10-16T08:38:44.663674+0000","last_unstale":"2023-10-16T08:39:58.403119+0000","last_undegraded":"2023-10-16T08:39:58.403119+0000","last_fullsized":"2023-10-16T08:39:58.403119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2023-10-16T08:39:58.402923+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2023-10-16T08:39:58.402923+0000","last_clean_scrub_stamp":"2023-10-16T08:39:58.402923+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:13:31.588625+0000","scrub_duration":0.0069404790000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.965292+0000","last_change":"2023-10-16T08:38:45.680890+0000","last_active":"2023-10-16T08:39:26.965292+0000","last_peered":"2023-10-16T08:39:26.965292+0000","last_clean":"2023-10-16T08:39:26.965292+0000","last_became_active":"2023-10-16T08:38:45.680266+0000","last_became_peered":"2023-10-16T08:38:45.680266+0000","last_unstale":"2023-10-16T08:39:26.965292+0000","last_undegraded":"2023-10-16T08:39:26.965292+0000","last_fullsized":"2023-10-16T08:39:26.965292+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'3","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:01.344970+0000","last_change":"2023-10-16T08:40:01.345161+0000","last_active":"2023-10-16T08:40:01.344970+0000","last_peered":"2023-10-16T08:40:01.344970+0000","last_clean":"2023-10-16T08:40:01.344970+0000","last_became_active":"2023-10-16T08:38:44.652658+0000","last_became_peered":"2023-10-16T08:38:44.652658+0000","last_unstale":"2023-10-16T08:40:01.344970+0000","last_undegraded":"2023-10-16T08:40:01.344970+0000","last_fullsized":"2023-10-16T08:40:01.344970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2023-10-16T08:40:01.344742+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2023-10-16T08:40:01.344742+0000","last_clean_scrub_stamp":"2023-10-16T08:40:01.344742+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:12.182792+0000","scrub_duration":0.0070571119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.959482+0000","last_change":"2023-10-16T08:38:45.665041+0000","last_active":"2023-10-16T08:39:26.959482+0000","last_peered":"2023-10-16T08:39:26.959482+0000","last_clean":"2023-10-16T08:39:26.959482+0000","last_became_active":"2023-10-16T08:38:45.664274+0000","last_became_peered":"2023-10-16T08:38:45.664274+0000","last_unstale":"2023-10-16T08:39:26.959482+0000","last_undegraded":"2023-10-16T08:39:26.959482+0000","last_fullsized":"2023-10-16T08:39:26.959482+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:56.005164+0000","last_change":"2023-10-16T08:39:56.005280+0000","last_active":"2023-10-16T08:39:56.005164+0000","last_peered":"2023-10-16T08:39:56.005164+0000","last_clean":"2023-10-16T08:39:56.005164+0000","last_became_active":"2023-10-16T08:38:44.645000+0000","last_became_peered":"2023-10-16T08:38:44.645000+0000","last_unstale":"2023-10-16T08:39:56.005164+0000","last_undegraded":"2023-10-16T08:39:56.005164+0000","last_fullsized":"2023-10-16T08:39:56.005164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"19'1","last_scrub_stamp":"2023-10-16T08:39:56.004937+0000","last_deep_scrub":"19'1","last_deep_scrub_stamp":"2023-10-16T08:39:56.004937+0000","last_clean_scrub_stamp":"2023-10-16T08:39:56.004937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:42:14.304698+0000","scrub_duration":0.0037060180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.008575+0000","last_change":"2023-10-16T08:38:45.664252+0000","last_active":"2023-10-16T08:39:27.008575+0000","last_peered":"2023-10-16T08:39:27.008575+0000","last_clean":"2023-10-16T08:39:27.008575+0000","last_became_active":"2023-10-16T08:38:45.663538+0000","last_became_peered":"2023-10-16T08:38:45.663538+0000","last_unstale":"2023-10-16T08:39:27.008575+0000","last_undegraded":"2023-10-16T08:39:27.008575+0000","last_fullsized":"2023-10-16T08:39:27.008575+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'6","reported_seq":29,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:47.192216+0000","last_change":"2023-10-16T08:39:47.192267+0000","last_active":"2023-10-16T08:39:47.192216+0000","last_peered":"2023-10-16T08:39:47.192216+0000","last_clean":"2023-10-16T08:39:47.192216+0000","last_became_active":"2023-10-16T08:38:44.645854+0000","last_became_peered":"2023-10-16T08:38:44.645854+0000","last_unstale":"2023-10-16T08:39:47.192216+0000","last_undegraded":"2023-10-16T08:39:47.192216+0000","last_fullsized":"2023-10-16T08:39:47.192216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'6","last_scrub_stamp":"2023-10-16T08:39:47.192138+0000","last_deep_scrub":"22'6","last_deep_scrub_stamp":"2023-10-16T08:39:47.192138+0000","last_clean_scrub_stamp":"2023-10-16T08:39:47.192138+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:31:12.293068+0000","scrub_duration":0.0072348439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6065,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":12130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":70,"ondisk_log_size":70,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574870,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":827392,"data_stored":674262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":0.85599999999999998},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":1.0920000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":1.038}]},{"osd":2,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":1.006},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.93500000000000005}]},{"osd":3,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.80000000000000004},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":1.256}]},{"osd":4,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":1.135},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.90000000000000002}]},{"osd":5,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.1850000000000001},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":1.216}]},{"osd":6,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.97599999999999998},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":1.0649999999999999}]}]},{"osd":6,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":88579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.0389999999999999},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.1459999999999999}]},{"osd":1,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":0.91300000000000003}]},{"osd":2,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.79700000000000004}]},{"osd":3,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":0.96599999999999997},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.69499999999999995}]},{"osd":4,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.74199999999999999}]},{"osd":5,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.0680000000000001}]},{"osd":7,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.04,"5min":2.04,"15min":2.04},"last":1.226},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":0.83599999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":83402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":1.2649999999999999}]},{"osd":1,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.1639999999999999},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.998}]},{"osd":2,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.2390000000000001},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.875}]},{"osd":3,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.194}]},{"osd":4,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":1.071}]},{"osd":6,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.1220000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":1.038}]},{"osd":7,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":1.214},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.288}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":241664,"data_stored":83438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.98199999999999998},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.04}]},{"osd":1,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.70399999999999996}]},{"osd":2,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.76200000000000001},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":1.258}]},{"osd":3,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.84799999999999998},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.93000000000000005}]},{"osd":5,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":1.1359999999999999},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":1.2290000000000001}]},{"osd":6,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":1.101},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":1.341}]},{"osd":7,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":1.1719999999999999},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.153,"5min":2.153,"15min":2.153},"last":1.2030000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":88579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.878}]},{"osd":1,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.625},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.69199999999999995}]},{"osd":2,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.93799999999999994}]},{"osd":4,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":1.131}]},{"osd":5,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.0029999999999999},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":1.165}]},{"osd":6,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.0720000000000001},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.0569999999999999}]},{"osd":7,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.0980000000000001},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.028}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":83312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.94199999999999995}]},{"osd":1,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.85499999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.90000000000000002}]},{"osd":4,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.028},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":1.159}]},{"osd":5,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":1.075},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.3149999999999999}]},{"osd":6,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.3660000000000001},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":1.3979999999999999}]},{"osd":7,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":1.452},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.2210000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":83472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.84399999999999997}]},{"osd":2,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.77800000000000002}]},{"osd":3,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":1.091},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.65300000000000002}]},{"osd":4,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":1.0449999999999999}]},{"osd":5,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.0009999999999999},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.397}]},{"osd":6,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":0.97999999999999998},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.95099999999999996}]},{"osd":7,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.3029999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.5449999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":1.028}]}]},{"osd":0,"up_from":13,"seq":55834574870,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":835584,"data_stored":674318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.89900000000000002}]},{"osd":2,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.61199999999999999}]},{"osd":3,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.85799999999999998},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":0.80800000000000005}]},{"osd":4,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.998},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":1.038}]},{"osd":5,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":1.079},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.3089999999999999}]},{"osd":6,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.1779999999999999},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.2729999999999999}]},{"osd":7,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.214},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":1.383}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:40:09.122 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.122 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.123 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.123 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.123 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.124 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.124 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.125 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.125 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.126 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.126 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.126 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.127 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.127 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.128 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.128 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.129 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.129 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.130 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.130 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.130 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.131 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.131 INFO:tasks.ceph:pgid 2.15 last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.132 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.132 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.132 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.133 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.133 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.134 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.134 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.134 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.135 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.135 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.136 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.136 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.137 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.137 INFO:tasks.ceph:pgid 2.a last_scrub_stamp 2023-10-16T08:38:43.584292+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.137 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.138 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.138 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.139 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.139 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.140 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:09.140 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-16T08:40:13.353 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:40:13.356 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:40:29.142 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:40:29.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.278+0000 7f6379cbd640 1 Processor -- start 2023-10-16T08:40:29.280 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.278+0000 7f6379cbd640 1 -- start start 2023-10-16T08:40:29.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.278+0000 7f6379cbd640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f637415b2e0 0x7f63740cc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:29.282 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.278+0000 7f6379cbd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63740cfa20 0x7f63740b2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:29.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.278+0000 7f6379cbd640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f637406fc20 con 0x7f63740ccd70 2023-10-16T08:40:29.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.278+0000 7f6379cbd640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63740cb380 con 0x7f63740cfa20 2023-10-16T08:40:29.283 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.278+0000 7f6379cbd640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63740b3200 con 0x7f637415b2e0 2023-10-16T08:40:29.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.278+0000 7f6373fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63740cfa20 0x7f63740b2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:29.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.278+0000 7f6373fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f63740cfa20 0x7f63740b2480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34692/0 (socket says 172.21.15.40:34692) 2023-10-16T08:40:29.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.278+0000 7f6373fff640 1 -- 172.21.15.40:0/1934864833 learned_addr learned my addr 172.21.15.40:0/1934864833 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:40:29.284 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.278+0000 7f63737fe640 1 --2- 172.21.15.40:0/1934864833 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f637415b2e0 0x7f63740cc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:29.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.278+0000 7f6373fff640 1 -- 172.21.15.40:0/1934864833 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f637415b2e0 msgr2=0x7f63740cc830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:40:29.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.278+0000 7f6373fff640 1 --2- 172.21.15.40:0/1934864833 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f637415b2e0 0x7f63740cc830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:29.285 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.278+0000 7f6373fff640 1 -- 172.21.15.40:0/1934864833 >> v1:172.21.15.86:6789/0 conn(0x7f63740ccd70 legacy=0x7f63740cf310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:29.286 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.278+0000 7f6373fff640 1 -- 172.21.15.40:0/1934864833 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f637406fee0 con 0x7f63740cfa20 2023-10-16T08:40:29.290 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.282+0000 7f6373fff640 1 --2- 172.21.15.40:0/1934864833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740cfa20 0x7f63740b2480 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f636c009e50 tx=0x7f636c01ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73fc04287cfbac6c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:29.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.282+0000 7f63727fc640 1 -- 172.21.15.40:0/1934864833 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c0a58e0 con 0x7f63740cfa20 2023-10-16T08:40:29.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.282+0000 7f63727fc640 1 -- 172.21.15.40:0/1934864833 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f636c0a5a40 con 0x7f63740cfa20 2023-10-16T08:40:29.291 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.282+0000 7f63727fc640 1 -- 172.21.15.40:0/1934864833 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f636c0a5dd0 con 0x7f63740cfa20 2023-10-16T08:40:29.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.282+0000 7f6379cbd640 1 -- 172.21.15.40:0/1934864833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740cfa20 msgr2=0x7f63740b2480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:29.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.282+0000 7f6379cbd640 1 --2- 172.21.15.40:0/1934864833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740cfa20 0x7f63740b2480 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f636c009e50 tx=0x7f636c01ca10 comp rx=0 tx=0).stop 2023-10-16T08:40:29.292 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.282+0000 7f6379cbd640 1 -- 172.21.15.40:0/1934864833 shutdown_connections 2023-10-16T08:40:29.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.282+0000 7f6379cbd640 1 --2- 172.21.15.40:0/1934864833 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740cfa20 0x7f63740b2480 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:29.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.282+0000 7f6379cbd640 1 --2- 172.21.15.40:0/1934864833 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f637415b2e0 0x7f63740cc830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:29.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.282+0000 7f6379cbd640 1 -- 172.21.15.40:0/1934864833 >> 172.21.15.40:0/1934864833 conn(0x7f637406d700 msgr2=0x7f637406db10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:29.293 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.282+0000 7f6379cbd640 1 -- 172.21.15.40:0/1934864833 shutdown_connections 2023-10-16T08:40:29.294 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.282+0000 7f6379cbd640 1 -- 172.21.15.40:0/1934864833 wait complete. 2023-10-16T08:40:29.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.282+0000 7f6379cbd640 1 Processor -- start 2023-10-16T08:40:29.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.282+0000 7f6379cbd640 1 -- start start 2023-10-16T08:40:29.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.282+0000 7f6379cbd640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740ccd70 0x7f63741553c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:29.295 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.282+0000 7f6379cbd640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63740cfa20 0x7f6374150560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:29.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.282+0000 7f6379cbd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f637415d300 0x7f6374153880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:29.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.282+0000 7f6379cbd640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63741565b0 con 0x7f63740ccd70 2023-10-16T08:40:29.296 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.282+0000 7f6379cbd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6374150aa0 con 0x7f637415d300 2023-10-16T08:40:29.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.282+0000 7f6379cbd640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63741559a0 con 0x7f63740cfa20 2023-10-16T08:40:29.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f63737fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740ccd70 0x7f63741553c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:29.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f63737fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740ccd70 0x7f63741553c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:34694/0 (socket says 172.21.15.40:34694) 2023-10-16T08:40:29.297 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f63737fe640 1 -- 172.21.15.40:0/544530788 learned_addr learned my addr 172.21.15.40:0/544530788 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:40:29.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f6372ffd640 1 --2- 172.21.15.40:0/544530788 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63740cfa20 0x7f6374150560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:29.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f6373fff640 1 --2- 172.21.15.40:0/544530788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f637415d300 0x7f6374153880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:29.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f63737fe640 1 -- 172.21.15.40:0/544530788 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63740cfa20 msgr2=0x7f6374150560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:29.298 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f63737fe640 1 --2- 172.21.15.40:0/544530788 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63740cfa20 0x7f6374150560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f63737fe640 1 -- 172.21.15.40:0/544530788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f637415d300 msgr2=0x7f6374153880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f63737fe640 1 --2- 172.21.15.40:0/544530788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f637415d300 0x7f6374153880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:29.299 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f63737fe640 1 -- 172.21.15.40:0/544530788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f636c009b30 con 0x7f63740ccd70 2023-10-16T08:40:29.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f63737fe640 1 --2- 172.21.15.40:0/544530788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740ccd70 0x7f63741553c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f636400d5f0 tx=0x7f636400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:29.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f635bfff640 1 -- 172.21.15.40:0/544530788 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63640125f0 con 0x7f63740ccd70 2023-10-16T08:40:29.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f635bfff640 1 -- 172.21.15.40:0/544530788 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6364012750 con 0x7f63740ccd70 2023-10-16T08:40:29.300 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f6379cbd640 1 -- 172.21.15.40:0/544530788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6374155c80 con 0x7f63740ccd70 2023-10-16T08:40:29.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f635bfff640 1 -- 172.21.15.40:0/544530788 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6364012ae0 con 0x7f63740ccd70 2023-10-16T08:40:29.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f6379cbd640 1 -- 172.21.15.40:0/544530788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6374156240 con 0x7f63740ccd70 2023-10-16T08:40:29.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f6379cbd640 1 -- 172.21.15.40:0/544530788 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f637404e9c0 con 0x7f63740ccd70 2023-10-16T08:40:29.301 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f635bfff640 1 -- 172.21.15.40:0/544530788 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f6364012c40 con 0x7f63740ccd70 2023-10-16T08:40:29.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f635bfff640 1 --2- 172.21.15.40:0/544530788 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f63600664a0 0x7f6360068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:29.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f635bfff640 1 -- 172.21.15.40:0/544530788 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f6364084440 con 0x7f63740ccd70 2023-10-16T08:40:29.302 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.286+0000 7f6372ffd640 1 --2- 172.21.15.40:0/544530788 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f63600664a0 0x7f6360068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:29.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.290+0000 7f6372ffd640 1 --2- 172.21.15.40:0/544530788 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f63600664a0 0x7f6360068960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f637415f740 tx=0x7f6368009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:29.303 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.290+0000 7f635bfff640 1 -- 172.21.15.40:0/544530788 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f6364051830 con 0x7f63740ccd70 2023-10-16T08:40:29.450 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.446+0000 7f6379cbd640 1 -- 172.21.15.40:0/544530788 --> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63741566f0 con 0x7f63600664a0 2023-10-16T08:40:29.473 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.470+0000 7f635bfff640 1 -- 172.21.15.40:0/544530788 <== mgr.4111 v2:172.21.15.86:6832/1188970899 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288213 (secure 0 0 0) 0x7f6368001370 con 0x7f63600664a0 2023-10-16T08:40:29.475 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:40:29.477 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-16T08:40:29.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.474+0000 7f6379cbd640 1 -- 172.21.15.40:0/544530788 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f63600664a0 msgr2=0x7f6360068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:29.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.474+0000 7f6379cbd640 1 --2- 172.21.15.40:0/544530788 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f63600664a0 0x7f6360068960 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f637415f740 tx=0x7f6368009040 comp rx=0 tx=0).stop 2023-10-16T08:40:29.479 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.474+0000 7f6379cbd640 1 -- 172.21.15.40:0/544530788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740ccd70 msgr2=0x7f63741553c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:29.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.474+0000 7f6379cbd640 1 --2- 172.21.15.40:0/544530788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740ccd70 0x7f63741553c0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f636400d5f0 tx=0x7f636400dab0 comp rx=0 tx=0).stop 2023-10-16T08:40:29.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.474+0000 7f6379cbd640 1 -- 172.21.15.40:0/544530788 shutdown_connections 2023-10-16T08:40:29.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.474+0000 7f6379cbd640 1 --2- 172.21.15.40:0/544530788 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f637415d300 0x7f6374153880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:29.480 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.474+0000 7f6379cbd640 1 --2- 172.21.15.40:0/544530788 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f63740cfa20 0x7f6374150560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:29.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.474+0000 7f6379cbd640 1 --2- 172.21.15.40:0/544530788 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f63600664a0 0x7f6360068960 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:29.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.474+0000 7f6379cbd640 1 --2- 172.21.15.40:0/544530788 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f63740ccd70 0x7f63741553c0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:29.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.474+0000 7f6379cbd640 1 -- 172.21.15.40:0/544530788 >> 172.21.15.40:0/544530788 conn(0x7f637406d700 msgr2=0x7f63740d0990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:29.481 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.474+0000 7f6379cbd640 1 -- 172.21.15.40:0/544530788 shutdown_connections 2023-10-16T08:40:29.482 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:29.474+0000 7f6379cbd640 1 -- 172.21.15.40:0/544530788 wait complete. 2023-10-16T08:40:29.496 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":73,"stamp":"2023-10-16T08:40:28.974740+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596433,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":220,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":191,"ondisk_log_size":191,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219152,"kb_used_data":3024,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755568,"statfs":{"total":773094113280,"available":772869701632,"internally_reserved":0,"allocated":3096576,"data_stored":1859362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004051"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:14.566064+0000","last_change":"2023-10-16T08:40:14.566175+0000","last_active":"2023-10-16T08:40:14.566064+0000","last_peered":"2023-10-16T08:40:14.566064+0000","last_clean":"2023-10-16T08:40:14.566064+0000","last_became_active":"2023-10-16T08:39:26.962446+0000","last_became_peered":"2023-10-16T08:39:26.962446+0000","last_unstale":"2023-10-16T08:40:14.566064+0000","last_undegraded":"2023-10-16T08:40:14.566064+0000","last_fullsized":"2023-10-16T08:40:14.566064+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:14.565850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:14.565850+0000","last_clean_scrub_stamp":"2023-10-16T08:40:14.565850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:48.417251+0000","scrub_duration":0.0038884470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.007933+0000","last_change":"2023-10-16T08:38:45.657659+0000","last_active":"2023-10-16T08:39:27.007933+0000","last_peered":"2023-10-16T08:39:27.007933+0000","last_clean":"2023-10-16T08:39:27.007933+0000","last_became_active":"2023-10-16T08:38:45.656884+0000","last_became_peered":"2023-10-16T08:38:45.656884+0000","last_unstale":"2023-10-16T08:39:27.007933+0000","last_undegraded":"2023-10-16T08:39:27.007933+0000","last_fullsized":"2023-10-16T08:39:27.007933+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:06.160364+0000","last_change":"2023-10-16T08:40:06.160545+0000","last_active":"2023-10-16T08:40:06.160364+0000","last_peered":"2023-10-16T08:40:06.160364+0000","last_clean":"2023-10-16T08:40:06.160364+0000","last_became_active":"2023-10-16T08:38:45.629297+0000","last_became_peered":"2023-10-16T08:38:45.629297+0000","last_unstale":"2023-10-16T08:40:06.160364+0000","last_undegraded":"2023-10-16T08:40:06.160364+0000","last_fullsized":"2023-10-16T08:40:06.160364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:06.160109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:06.160109+0000","last_clean_scrub_stamp":"2023-10-16T08:40:06.160109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:29.917626+0000","scrub_duration":0.003797534,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:10.594378+0000","last_change":"2023-10-16T08:40:10.594472+0000","last_active":"2023-10-16T08:40:10.594378+0000","last_peered":"2023-10-16T08:40:10.594378+0000","last_clean":"2023-10-16T08:40:10.594378+0000","last_became_active":"2023-10-16T08:38:45.675327+0000","last_became_peered":"2023-10-16T08:38:45.675327+0000","last_unstale":"2023-10-16T08:40:10.594378+0000","last_undegraded":"2023-10-16T08:40:10.594378+0000","last_fullsized":"2023-10-16T08:40:10.594378+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:10.594149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:10.594149+0000","last_clean_scrub_stamp":"2023-10-16T08:40:10.594149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:06:49.151485+0000","scrub_duration":0.003827324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:10.247793+0000","last_change":"2023-10-16T08:40:10.247892+0000","last_active":"2023-10-16T08:40:10.247793+0000","last_peered":"2023-10-16T08:40:10.247793+0000","last_clean":"2023-10-16T08:40:10.247793+0000","last_became_active":"2023-10-16T08:38:45.652808+0000","last_became_peered":"2023-10-16T08:38:45.652808+0000","last_unstale":"2023-10-16T08:40:10.247793+0000","last_undegraded":"2023-10-16T08:40:10.247793+0000","last_fullsized":"2023-10-16T08:40:10.247793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:10.247511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:10.247511+0000","last_clean_scrub_stamp":"2023-10-16T08:40:10.247511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:29:35.626689+0000","scrub_duration":0.0039935719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:17.611393+0000","last_change":"2023-10-16T08:40:17.611491+0000","last_active":"2023-10-16T08:40:17.611393+0000","last_peered":"2023-10-16T08:40:17.611393+0000","last_clean":"2023-10-16T08:40:17.611393+0000","last_became_active":"2023-10-16T08:38:45.671146+0000","last_became_peered":"2023-10-16T08:38:45.671146+0000","last_unstale":"2023-10-16T08:40:17.611393+0000","last_undegraded":"2023-10-16T08:40:17.611393+0000","last_fullsized":"2023-10-16T08:40:17.611393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:17.611169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:17.611169+0000","last_clean_scrub_stamp":"2023-10-16T08:40:17.611169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:37.226826+0000","scrub_duration":0.0038744809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:25.379392+0000","last_change":"2023-10-16T08:40:25.379550+0000","last_active":"2023-10-16T08:40:25.379392+0000","last_peered":"2023-10-16T08:40:25.379392+0000","last_clean":"2023-10-16T08:40:25.379392+0000","last_became_active":"2023-10-16T08:38:45.651848+0000","last_became_peered":"2023-10-16T08:38:45.651848+0000","last_unstale":"2023-10-16T08:40:25.379392+0000","last_undegraded":"2023-10-16T08:40:25.379392+0000","last_fullsized":"2023-10-16T08:40:25.379392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:25.379078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:25.379078+0000","last_clean_scrub_stamp":"2023-10-16T08:40:25.379078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:46:42.710112+0000","scrub_duration":0.0039533019999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:06.819154+0000","last_change":"2023-10-16T08:40:06.819332+0000","last_active":"2023-10-16T08:40:06.819154+0000","last_peered":"2023-10-16T08:40:06.819154+0000","last_clean":"2023-10-16T08:40:06.819154+0000","last_became_active":"2023-10-16T08:38:45.664702+0000","last_became_peered":"2023-10-16T08:38:45.664702+0000","last_unstale":"2023-10-16T08:40:06.819154+0000","last_undegraded":"2023-10-16T08:40:06.819154+0000","last_fullsized":"2023-10-16T08:40:06.819154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:06.818902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:06.818902+0000","last_clean_scrub_stamp":"2023-10-16T08:40:06.818902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:00.868499+0000","scrub_duration":0.0035812309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:13.209914+0000","last_change":"2023-10-16T08:40:13.210016+0000","last_active":"2023-10-16T08:40:13.209914+0000","last_peered":"2023-10-16T08:40:13.209914+0000","last_clean":"2023-10-16T08:40:13.209914+0000","last_became_active":"2023-10-16T08:38:45.666474+0000","last_became_peered":"2023-10-16T08:38:45.666474+0000","last_unstale":"2023-10-16T08:40:13.209914+0000","last_undegraded":"2023-10-16T08:40:13.209914+0000","last_fullsized":"2023-10-16T08:40:13.209914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:13.209675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:13.209675+0000","last_clean_scrub_stamp":"2023-10-16T08:40:13.209675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:06:51.767012+0000","scrub_duration":0.0033902530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:15.995400+0000","last_change":"2023-10-16T08:40:15.995499+0000","last_active":"2023-10-16T08:40:15.995400+0000","last_peered":"2023-10-16T08:40:15.995400+0000","last_clean":"2023-10-16T08:40:15.995400+0000","last_became_active":"2023-10-16T08:38:45.668278+0000","last_became_peered":"2023-10-16T08:38:45.668278+0000","last_unstale":"2023-10-16T08:40:15.995400+0000","last_undegraded":"2023-10-16T08:40:15.995400+0000","last_fullsized":"2023-10-16T08:40:15.995400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:15.995163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:15.995163+0000","last_clean_scrub_stamp":"2023-10-16T08:40:15.995163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:04:59.725405+0000","scrub_duration":0.0041064329999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:12.005283+0000","last_change":"2023-10-16T08:40:12.005399+0000","last_active":"2023-10-16T08:40:12.005283+0000","last_peered":"2023-10-16T08:40:12.005283+0000","last_clean":"2023-10-16T08:40:12.005283+0000","last_became_active":"2023-10-16T08:38:45.663813+0000","last_became_peered":"2023-10-16T08:38:45.663813+0000","last_unstale":"2023-10-16T08:40:12.005283+0000","last_undegraded":"2023-10-16T08:40:12.005283+0000","last_fullsized":"2023-10-16T08:40:12.005283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:12.005041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:12.005041+0000","last_clean_scrub_stamp":"2023-10-16T08:40:12.005041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:17.822987+0000","scrub_duration":0.0037935099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:58.863583+0000","last_change":"2023-10-16T08:39:58.863718+0000","last_active":"2023-10-16T08:39:58.863583+0000","last_peered":"2023-10-16T08:39:58.863583+0000","last_clean":"2023-10-16T08:39:58.863583+0000","last_became_active":"2023-10-16T08:38:45.675320+0000","last_became_peered":"2023-10-16T08:38:45.675320+0000","last_unstale":"2023-10-16T08:39:58.863583+0000","last_undegraded":"2023-10-16T08:39:58.863583+0000","last_fullsized":"2023-10-16T08:39:58.863583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:58.863355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:58.863355+0000","last_clean_scrub_stamp":"2023-10-16T08:39:58.863355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:17:12.041514+0000","scrub_duration":0.0033670039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:19.634096+0000","last_change":"2023-10-16T08:40:19.634287+0000","last_active":"2023-10-16T08:40:19.634096+0000","last_peered":"2023-10-16T08:40:19.634096+0000","last_clean":"2023-10-16T08:40:19.634096+0000","last_became_active":"2023-10-16T08:38:45.667428+0000","last_became_peered":"2023-10-16T08:38:45.667428+0000","last_unstale":"2023-10-16T08:40:19.634096+0000","last_undegraded":"2023-10-16T08:40:19.634096+0000","last_fullsized":"2023-10-16T08:40:19.634096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:19.633788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:19.633788+0000","last_clean_scrub_stamp":"2023-10-16T08:40:19.633788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:10:51.712948+0000","scrub_duration":0.0043199290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:50.221350+0000","last_change":"2023-10-16T08:39:50.221441+0000","last_active":"2023-10-16T08:39:50.221350+0000","last_peered":"2023-10-16T08:39:50.221350+0000","last_clean":"2023-10-16T08:39:50.221350+0000","last_became_active":"2023-10-16T08:38:45.636785+0000","last_became_peered":"2023-10-16T08:38:45.636785+0000","last_unstale":"2023-10-16T08:39:50.221350+0000","last_undegraded":"2023-10-16T08:39:50.221350+0000","last_fullsized":"2023-10-16T08:39:50.221350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:50.221128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:50.221128+0000","last_clean_scrub_stamp":"2023-10-16T08:39:50.221128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:58:43.521146+0000","scrub_duration":0.002811554,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:20.751761+0000","last_change":"2023-10-16T08:40:20.751864+0000","last_active":"2023-10-16T08:40:20.751761+0000","last_peered":"2023-10-16T08:40:20.751761+0000","last_clean":"2023-10-16T08:40:20.751761+0000","last_became_active":"2023-10-16T08:38:45.655395+0000","last_became_peered":"2023-10-16T08:38:45.655395+0000","last_unstale":"2023-10-16T08:40:20.751761+0000","last_undegraded":"2023-10-16T08:40:20.751761+0000","last_fullsized":"2023-10-16T08:40:20.751761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:20.751504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:20.751504+0000","last_clean_scrub_stamp":"2023-10-16T08:40:20.751504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:14.801101+0000","scrub_duration":0.0037562749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:04.505262+0000","last_change":"2023-10-16T08:40:04.505370+0000","last_active":"2023-10-16T08:40:04.505262+0000","last_peered":"2023-10-16T08:40:04.505262+0000","last_clean":"2023-10-16T08:40:04.505262+0000","last_became_active":"2023-10-16T08:38:45.663312+0000","last_became_peered":"2023-10-16T08:38:45.663312+0000","last_unstale":"2023-10-16T08:40:04.505262+0000","last_undegraded":"2023-10-16T08:40:04.505262+0000","last_fullsized":"2023-10-16T08:40:04.505262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:04.505050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:04.505050+0000","last_clean_scrub_stamp":"2023-10-16T08:40:04.505050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:15.343100+0000","scrub_duration":0.0035649169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:13.571796+0000","last_change":"2023-10-16T08:40:13.571890+0000","last_active":"2023-10-16T08:40:13.571796+0000","last_peered":"2023-10-16T08:40:13.571796+0000","last_clean":"2023-10-16T08:40:13.571796+0000","last_became_active":"2023-10-16T08:39:27.012068+0000","last_became_peered":"2023-10-16T08:39:27.012068+0000","last_unstale":"2023-10-16T08:40:13.571796+0000","last_undegraded":"2023-10-16T08:40:13.571796+0000","last_fullsized":"2023-10-16T08:40:13.571796+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:13.571587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:13.571587+0000","last_clean_scrub_stamp":"2023-10-16T08:40:13.571587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:55:53.091091+0000","scrub_duration":0.0036468709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:05.650748+0000","last_change":"2023-10-16T08:40:05.650848+0000","last_active":"2023-10-16T08:40:05.650748+0000","last_peered":"2023-10-16T08:40:05.650748+0000","last_clean":"2023-10-16T08:40:05.650748+0000","last_became_active":"2023-10-16T08:38:45.630641+0000","last_became_peered":"2023-10-16T08:38:45.630641+0000","last_unstale":"2023-10-16T08:40:05.650748+0000","last_undegraded":"2023-10-16T08:40:05.650748+0000","last_fullsized":"2023-10-16T08:40:05.650748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:05.650492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:05.650492+0000","last_clean_scrub_stamp":"2023-10-16T08:40:05.650492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:42:23.950253+0000","scrub_duration":0.0038292840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.010376+0000","last_change":"2023-10-16T08:38:45.650087+0000","last_active":"2023-10-16T08:39:27.010376+0000","last_peered":"2023-10-16T08:39:27.010376+0000","last_clean":"2023-10-16T08:39:27.010376+0000","last_became_active":"2023-10-16T08:38:45.649475+0000","last_became_peered":"2023-10-16T08:38:45.649475+0000","last_unstale":"2023-10-16T08:39:27.010376+0000","last_undegraded":"2023-10-16T08:39:27.010376+0000","last_fullsized":"2023-10-16T08:39:27.010376+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:18.946495+0000","last_change":"2023-10-16T08:40:18.946701+0000","last_active":"2023-10-16T08:40:18.946495+0000","last_peered":"2023-10-16T08:40:18.946495+0000","last_clean":"2023-10-16T08:40:18.946495+0000","last_became_active":"2023-10-16T08:38:45.665291+0000","last_became_peered":"2023-10-16T08:38:45.665291+0000","last_unstale":"2023-10-16T08:40:18.946495+0000","last_undegraded":"2023-10-16T08:40:18.946495+0000","last_fullsized":"2023-10-16T08:40:18.946495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:18.946250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:18.946250+0000","last_clean_scrub_stamp":"2023-10-16T08:40:18.946250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:46:43.621078+0000","scrub_duration":0.00352601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:04.141444+0000","last_change":"2023-10-16T08:40:04.141546+0000","last_active":"2023-10-16T08:40:04.141444+0000","last_peered":"2023-10-16T08:40:04.141444+0000","last_clean":"2023-10-16T08:40:04.141444+0000","last_became_active":"2023-10-16T08:38:45.652412+0000","last_became_peered":"2023-10-16T08:38:45.652412+0000","last_unstale":"2023-10-16T08:40:04.141444+0000","last_undegraded":"2023-10-16T08:40:04.141444+0000","last_fullsized":"2023-10-16T08:40:04.141444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:04.141224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:04.141224+0000","last_clean_scrub_stamp":"2023-10-16T08:40:04.141224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:07.519281+0000","scrub_duration":0.0035640720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:05.502168+0000","last_change":"2023-10-16T08:40:05.502348+0000","last_active":"2023-10-16T08:40:05.502168+0000","last_peered":"2023-10-16T08:40:05.502168+0000","last_clean":"2023-10-16T08:40:05.502168+0000","last_became_active":"2023-10-16T08:38:45.676844+0000","last_became_peered":"2023-10-16T08:38:45.676844+0000","last_unstale":"2023-10-16T08:40:05.502168+0000","last_undegraded":"2023-10-16T08:40:05.502168+0000","last_fullsized":"2023-10-16T08:40:05.502168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:05.501915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:05.501915+0000","last_clean_scrub_stamp":"2023-10-16T08:40:05.501915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:46:30.176743+0000","scrub_duration":0.0034663509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:05.864255+0000","last_change":"2023-10-16T08:40:05.864353+0000","last_active":"2023-10-16T08:40:05.864255+0000","last_peered":"2023-10-16T08:40:05.864255+0000","last_clean":"2023-10-16T08:40:05.864255+0000","last_became_active":"2023-10-16T08:38:45.662944+0000","last_became_peered":"2023-10-16T08:38:45.662944+0000","last_unstale":"2023-10-16T08:40:05.864255+0000","last_undegraded":"2023-10-16T08:40:05.864255+0000","last_fullsized":"2023-10-16T08:40:05.864255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:05.864043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:05.864043+0000","last_clean_scrub_stamp":"2023-10-16T08:40:05.864043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:39.715444+0000","scrub_duration":0.0038577170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:00.546243+0000","last_change":"2023-10-16T08:40:00.546291+0000","last_active":"2023-10-16T08:40:00.546243+0000","last_peered":"2023-10-16T08:40:00.546243+0000","last_clean":"2023-10-16T08:40:00.546243+0000","last_became_active":"2023-10-16T08:38:45.666641+0000","last_became_peered":"2023-10-16T08:38:45.666641+0000","last_unstale":"2023-10-16T08:40:00.546243+0000","last_undegraded":"2023-10-16T08:40:00.546243+0000","last_fullsized":"2023-10-16T08:40:00.546243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:00.546142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:00.546142+0000","last_clean_scrub_stamp":"2023-10-16T08:40:00.546142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:13:33.731843+0000","scrub_duration":0.0032222819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.008655+0000","last_change":"2023-10-16T08:38:45.670526+0000","last_active":"2023-10-16T08:39:27.008655+0000","last_peered":"2023-10-16T08:39:27.008655+0000","last_clean":"2023-10-16T08:39:27.008655+0000","last_became_active":"2023-10-16T08:38:45.669734+0000","last_became_peered":"2023-10-16T08:38:45.669734+0000","last_unstale":"2023-10-16T08:39:27.008655+0000","last_undegraded":"2023-10-16T08:39:27.008655+0000","last_fullsized":"2023-10-16T08:39:27.008655+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:07.003998+0000","last_change":"2023-10-16T08:40:07.004096+0000","last_active":"2023-10-16T08:40:07.003998+0000","last_peered":"2023-10-16T08:40:07.003998+0000","last_clean":"2023-10-16T08:40:07.003998+0000","last_became_active":"2023-10-16T08:38:45.665972+0000","last_became_peered":"2023-10-16T08:38:45.665972+0000","last_unstale":"2023-10-16T08:40:07.003998+0000","last_undegraded":"2023-10-16T08:40:07.003998+0000","last_fullsized":"2023-10-16T08:40:07.003998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:07.003796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:07.003796+0000","last_clean_scrub_stamp":"2023-10-16T08:40:07.003796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:01.053394+0000","scrub_duration":0.0034323180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.009996+0000","last_change":"2023-10-16T08:38:45.667047+0000","last_active":"2023-10-16T08:39:27.009996+0000","last_peered":"2023-10-16T08:39:27.009996+0000","last_clean":"2023-10-16T08:39:27.009996+0000","last_became_active":"2023-10-16T08:38:45.666403+0000","last_became_peered":"2023-10-16T08:38:45.666403+0000","last_unstale":"2023-10-16T08:39:27.009996+0000","last_undegraded":"2023-10-16T08:39:27.009996+0000","last_fullsized":"2023-10-16T08:39:27.009996+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:04.840228+0000","last_change":"2023-10-16T08:40:04.840318+0000","last_active":"2023-10-16T08:40:04.840228+0000","last_peered":"2023-10-16T08:40:04.840228+0000","last_clean":"2023-10-16T08:40:04.840228+0000","last_became_active":"2023-10-16T08:38:45.664572+0000","last_became_peered":"2023-10-16T08:38:45.664572+0000","last_unstale":"2023-10-16T08:40:04.840228+0000","last_undegraded":"2023-10-16T08:40:04.840228+0000","last_fullsized":"2023-10-16T08:40:04.840228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:04.840012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:04.840012+0000","last_clean_scrub_stamp":"2023-10-16T08:40:04.840012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:55:44.359515+0000","scrub_duration":0.0038448359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:14.056731+0000","last_change":"2023-10-16T08:40:14.056845+0000","last_active":"2023-10-16T08:40:14.056731+0000","last_peered":"2023-10-16T08:40:14.056731+0000","last_clean":"2023-10-16T08:40:14.056731+0000","last_became_active":"2023-10-16T08:38:45.666547+0000","last_became_peered":"2023-10-16T08:38:45.666547+0000","last_unstale":"2023-10-16T08:40:14.056731+0000","last_undegraded":"2023-10-16T08:40:14.056731+0000","last_fullsized":"2023-10-16T08:40:14.056731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:14.056503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:14.056503+0000","last_clean_scrub_stamp":"2023-10-16T08:40:14.056503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:08.106100+0000","scrub_duration":0.0033984269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:11.524811+0000","last_change":"2023-10-16T08:40:11.524934+0000","last_active":"2023-10-16T08:40:11.524811+0000","last_peered":"2023-10-16T08:40:11.524811+0000","last_clean":"2023-10-16T08:40:11.524811+0000","last_became_active":"2023-10-16T08:39:27.009983+0000","last_became_peered":"2023-10-16T08:39:27.009983+0000","last_unstale":"2023-10-16T08:40:11.524811+0000","last_undegraded":"2023-10-16T08:40:11.524811+0000","last_fullsized":"2023-10-16T08:40:11.524811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:11.524600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:11.524600+0000","last_clean_scrub_stamp":"2023-10-16T08:40:11.524600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:50:05.973297+0000","scrub_duration":0.003632943,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:58.646552+0000","last_change":"2023-10-16T08:39:58.646655+0000","last_active":"2023-10-16T08:39:58.646552+0000","last_peered":"2023-10-16T08:39:58.646552+0000","last_clean":"2023-10-16T08:39:58.646552+0000","last_became_active":"2023-10-16T08:39:27.013658+0000","last_became_peered":"2023-10-16T08:39:27.013658+0000","last_unstale":"2023-10-16T08:39:58.646552+0000","last_undegraded":"2023-10-16T08:39:58.646552+0000","last_fullsized":"2023-10-16T08:39:58.646552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:58.646339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:58.646339+0000","last_clean_scrub_stamp":"2023-10-16T08:39:58.646339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:32:08.951921+0000","scrub_duration":0.0036955680000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:02.119406+0000","last_change":"2023-10-16T08:40:02.119578+0000","last_active":"2023-10-16T08:40:02.119406+0000","last_peered":"2023-10-16T08:40:02.119406+0000","last_clean":"2023-10-16T08:40:02.119406+0000","last_became_active":"2023-10-16T08:38:45.636762+0000","last_became_peered":"2023-10-16T08:38:45.636762+0000","last_unstale":"2023-10-16T08:40:02.119406+0000","last_undegraded":"2023-10-16T08:40:02.119406+0000","last_fullsized":"2023-10-16T08:40:02.119406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:02.119151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:02.119151+0000","last_clean_scrub_stamp":"2023-10-16T08:40:02.119151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:23:40.725505+0000","scrub_duration":0.00382139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:52.785339+0000","last_change":"2023-10-16T08:39:52.785439+0000","last_active":"2023-10-16T08:39:52.785339+0000","last_peered":"2023-10-16T08:39:52.785339+0000","last_clean":"2023-10-16T08:39:52.785339+0000","last_became_active":"2023-10-16T08:38:45.658808+0000","last_became_peered":"2023-10-16T08:38:45.658808+0000","last_unstale":"2023-10-16T08:39:52.785339+0000","last_undegraded":"2023-10-16T08:39:52.785339+0000","last_fullsized":"2023-10-16T08:39:52.785339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:52.785111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:52.785111+0000","last_clean_scrub_stamp":"2023-10-16T08:39:52.785111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:48.998080+0000","scrub_duration":0.0036557069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:48.153116+0000","last_change":"2023-10-16T08:39:48.153181+0000","last_active":"2023-10-16T08:39:48.153116+0000","last_peered":"2023-10-16T08:39:48.153116+0000","last_clean":"2023-10-16T08:39:48.153116+0000","last_became_active":"2023-10-16T08:38:44.643614+0000","last_became_peered":"2023-10-16T08:38:44.643614+0000","last_unstale":"2023-10-16T08:39:48.153116+0000","last_undegraded":"2023-10-16T08:39:48.153116+0000","last_fullsized":"2023-10-16T08:39:48.153116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"19'1","last_scrub_stamp":"2023-10-16T08:39:48.153004+0000","last_deep_scrub":"19'1","last_deep_scrub_stamp":"2023-10-16T08:39:48.153004+0000","last_clean_scrub_stamp":"2023-10-16T08:39:48.153004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:20.737033+0000","scrub_duration":0.002884073,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:26.791436+0000","last_change":"2023-10-16T08:40:26.791622+0000","last_active":"2023-10-16T08:40:26.791436+0000","last_peered":"2023-10-16T08:40:26.791436+0000","last_clean":"2023-10-16T08:40:26.791436+0000","last_became_active":"2023-10-16T08:38:45.658409+0000","last_became_peered":"2023-10-16T08:38:45.658409+0000","last_unstale":"2023-10-16T08:40:26.791436+0000","last_undegraded":"2023-10-16T08:40:26.791436+0000","last_fullsized":"2023-10-16T08:40:26.791436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:26.791165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:26.791165+0000","last_clean_scrub_stamp":"2023-10-16T08:40:26.791165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:37.629215+0000","scrub_duration":0.0038734239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:00.189167+0000","last_change":"2023-10-16T08:40:00.189264+0000","last_active":"2023-10-16T08:40:00.189167+0000","last_peered":"2023-10-16T08:40:00.189167+0000","last_clean":"2023-10-16T08:40:00.189167+0000","last_became_active":"2023-10-16T08:38:44.658257+0000","last_became_peered":"2023-10-16T08:38:44.658257+0000","last_unstale":"2023-10-16T08:40:00.189167+0000","last_undegraded":"2023-10-16T08:40:00.189167+0000","last_fullsized":"2023-10-16T08:40:00.189167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"19'1","last_scrub_stamp":"2023-10-16T08:40:00.188936+0000","last_deep_scrub":"19'1","last_deep_scrub_stamp":"2023-10-16T08:40:00.188936+0000","last_clean_scrub_stamp":"2023-10-16T08:40:00.188936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:03.566993+0000","scrub_duration":0.0036410499999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:51.092373+0000","last_change":"2023-10-16T08:39:51.092469+0000","last_active":"2023-10-16T08:39:51.092373+0000","last_peered":"2023-10-16T08:39:51.092373+0000","last_clean":"2023-10-16T08:39:51.092373+0000","last_became_active":"2023-10-16T08:38:44.645931+0000","last_became_peered":"2023-10-16T08:38:44.645931+0000","last_unstale":"2023-10-16T08:39:51.092373+0000","last_undegraded":"2023-10-16T08:39:51.092373+0000","last_fullsized":"2023-10-16T08:39:51.092373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:51.092105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:51.092105+0000","last_clean_scrub_stamp":"2023-10-16T08:39:51.092105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:32:01.397687+0000","scrub_duration":0.0039256869999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:01.915584+0000","last_change":"2023-10-16T08:40:01.915678+0000","last_active":"2023-10-16T08:40:01.915584+0000","last_peered":"2023-10-16T08:40:01.915584+0000","last_clean":"2023-10-16T08:40:01.915584+0000","last_became_active":"2023-10-16T08:38:45.662072+0000","last_became_peered":"2023-10-16T08:38:45.662072+0000","last_unstale":"2023-10-16T08:40:01.915584+0000","last_undegraded":"2023-10-16T08:40:01.915584+0000","last_fullsized":"2023-10-16T08:40:01.915584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:01.915369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:01.915369+0000","last_clean_scrub_stamp":"2023-10-16T08:40:01.915369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:49:56.364066+0000","scrub_duration":0.0034239230000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:50.531962+0000","last_change":"2023-10-16T08:39:50.532059+0000","last_active":"2023-10-16T08:39:50.531962+0000","last_peered":"2023-10-16T08:39:50.531962+0000","last_clean":"2023-10-16T08:39:50.531962+0000","last_became_active":"2023-10-16T08:38:44.657065+0000","last_became_peered":"2023-10-16T08:38:44.657065+0000","last_unstale":"2023-10-16T08:39:50.531962+0000","last_undegraded":"2023-10-16T08:39:50.531962+0000","last_fullsized":"2023-10-16T08:39:50.531962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:50.531739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:50.531739+0000","last_clean_scrub_stamp":"2023-10-16T08:39:50.531739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:49:44.980436+0000","scrub_duration":0.0034347560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.010905+0000","last_change":"2023-10-16T08:38:45.668048+0000","last_active":"2023-10-16T08:39:27.010905+0000","last_peered":"2023-10-16T08:39:27.010905+0000","last_clean":"2023-10-16T08:39:27.010905+0000","last_became_active":"2023-10-16T08:38:45.667413+0000","last_became_peered":"2023-10-16T08:38:45.667413+0000","last_unstale":"2023-10-16T08:39:27.010905+0000","last_undegraded":"2023-10-16T08:39:27.010905+0000","last_fullsized":"2023-10-16T08:39:27.010905+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:53.126900+0000","last_change":"2023-10-16T08:39:53.127012+0000","last_active":"2023-10-16T08:39:53.126900+0000","last_peered":"2023-10-16T08:39:53.126900+0000","last_clean":"2023-10-16T08:39:53.126900+0000","last_became_active":"2023-10-16T08:38:44.662383+0000","last_became_peered":"2023-10-16T08:38:44.662383+0000","last_unstale":"2023-10-16T08:39:53.126900+0000","last_undegraded":"2023-10-16T08:39:53.126900+0000","last_fullsized":"2023-10-16T08:39:53.126900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:53.126641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:53.126641+0000","last_clean_scrub_stamp":"2023-10-16T08:39:53.126641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:32:03.432222+0000","scrub_duration":0.0040293320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:15.602153+0000","last_change":"2023-10-16T08:40:15.602249+0000","last_active":"2023-10-16T08:40:15.602153+0000","last_peered":"2023-10-16T08:40:15.602153+0000","last_clean":"2023-10-16T08:40:15.602153+0000","last_became_active":"2023-10-16T08:38:45.678050+0000","last_became_peered":"2023-10-16T08:38:45.678050+0000","last_unstale":"2023-10-16T08:40:15.602153+0000","last_undegraded":"2023-10-16T08:40:15.602153+0000","last_fullsized":"2023-10-16T08:40:15.602153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:15.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:15.601899+0000","last_clean_scrub_stamp":"2023-10-16T08:40:15.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:19:06.907714+0000","scrub_duration":0.003337497,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:51.509756+0000","last_change":"2023-10-16T08:39:51.509855+0000","last_active":"2023-10-16T08:39:51.509756+0000","last_peered":"2023-10-16T08:39:51.509756+0000","last_clean":"2023-10-16T08:39:51.509756+0000","last_became_active":"2023-10-16T08:38:44.652618+0000","last_became_peered":"2023-10-16T08:38:44.652618+0000","last_unstale":"2023-10-16T08:39:51.509756+0000","last_undegraded":"2023-10-16T08:39:51.509756+0000","last_fullsized":"2023-10-16T08:39:51.509756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:51.509535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:51.509535+0000","last_clean_scrub_stamp":"2023-10-16T08:39:51.509535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:55:31.029038+0000","scrub_duration":0.0032865199999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:26.959627+0000","last_change":"2023-10-16T08:38:45.656603+0000","last_active":"2023-10-16T08:39:26.959627+0000","last_peered":"2023-10-16T08:39:26.959627+0000","last_clean":"2023-10-16T08:39:26.959627+0000","last_became_active":"2023-10-16T08:38:45.655213+0000","last_became_peered":"2023-10-16T08:38:45.655213+0000","last_unstale":"2023-10-16T08:39:26.959627+0000","last_undegraded":"2023-10-16T08:39:26.959627+0000","last_fullsized":"2023-10-16T08:39:26.959627+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'5","reported_seq":28,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:11.675792+0000","last_change":"2023-10-16T08:40:11.675907+0000","last_active":"2023-10-16T08:40:11.675792+0000","last_peered":"2023-10-16T08:40:11.675792+0000","last_clean":"2023-10-16T08:40:11.675792+0000","last_became_active":"2023-10-16T08:38:44.651359+0000","last_became_peered":"2023-10-16T08:38:44.651359+0000","last_unstale":"2023-10-16T08:40:11.675792+0000","last_undegraded":"2023-10-16T08:40:11.675792+0000","last_fullsized":"2023-10-16T08:40:11.675792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'5","last_scrub_stamp":"2023-10-16T08:40:11.675566+0000","last_deep_scrub":"22'5","last_deep_scrub_stamp":"2023-10-16T08:40:11.675566+0000","last_clean_scrub_stamp":"2023-10-16T08:40:11.675566+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:23:50.281919+0000","scrub_duration":0.0091403179999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:54.792955+0000","last_change":"2023-10-16T08:39:54.793071+0000","last_active":"2023-10-16T08:39:54.792955+0000","last_peered":"2023-10-16T08:39:54.792955+0000","last_clean":"2023-10-16T08:39:54.792955+0000","last_became_active":"2023-10-16T08:38:45.660611+0000","last_became_peered":"2023-10-16T08:38:45.660611+0000","last_unstale":"2023-10-16T08:39:54.792955+0000","last_undegraded":"2023-10-16T08:39:54.792955+0000","last_fullsized":"2023-10-16T08:39:54.792955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:54.792721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:54.792721+0000","last_clean_scrub_stamp":"2023-10-16T08:39:54.792721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:27.376750+0000","scrub_duration":0.0038456089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'20","reported_seq":43,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:59.159576+0000","last_change":"2023-10-16T08:39:59.159729+0000","last_active":"2023-10-16T08:39:59.159576+0000","last_peered":"2023-10-16T08:39:59.159576+0000","last_clean":"2023-10-16T08:39:59.159576+0000","last_became_active":"2023-10-16T08:38:44.647001+0000","last_became_peered":"2023-10-16T08:38:44.647001+0000","last_unstale":"2023-10-16T08:39:59.159576+0000","last_undegraded":"2023-10-16T08:39:59.159576+0000","last_fullsized":"2023-10-16T08:39:59.159576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'20","last_scrub_stamp":"2023-10-16T08:39:59.159359+0000","last_deep_scrub":"22'20","last_deep_scrub_stamp":"2023-10-16T08:39:59.159359+0000","last_clean_scrub_stamp":"2023-10-16T08:39:59.159359+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:42:17.459120+0000","scrub_duration":0.0099116120000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5267,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:09.284560+0000","last_change":"2023-10-16T08:40:09.284656+0000","last_active":"2023-10-16T08:40:09.284560+0000","last_peered":"2023-10-16T08:40:09.284560+0000","last_clean":"2023-10-16T08:40:09.284560+0000","last_became_active":"2023-10-16T08:38:45.667334+0000","last_became_peered":"2023-10-16T08:38:45.667334+0000","last_unstale":"2023-10-16T08:40:09.284560+0000","last_undegraded":"2023-10-16T08:40:09.284560+0000","last_fullsized":"2023-10-16T08:40:09.284560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:09.284344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:09.284344+0000","last_clean_scrub_stamp":"2023-10-16T08:40:09.284344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:37:59.060122+0000","scrub_duration":0.0038307739999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'4","reported_seq":27,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:46.198227+0000","last_change":"2023-10-16T08:39:46.198310+0000","last_active":"2023-10-16T08:39:46.198227+0000","last_peered":"2023-10-16T08:39:46.198227+0000","last_clean":"2023-10-16T08:39:46.198227+0000","last_became_active":"2023-10-16T08:38:44.655469+0000","last_became_peered":"2023-10-16T08:38:44.655469+0000","last_unstale":"2023-10-16T08:39:46.198227+0000","last_undegraded":"2023-10-16T08:39:46.198227+0000","last_fullsized":"2023-10-16T08:39:46.198227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4","last_scrub_stamp":"2023-10-16T08:39:46.198102+0000","last_deep_scrub":"22'4","last_deep_scrub_stamp":"2023-10-16T08:39:46.198102+0000","last_clean_scrub_stamp":"2023-10-16T08:39:46.198102+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:19:00.034900+0000","scrub_duration":0.00664833,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:12.071405+0000","last_change":"2023-10-16T08:40:12.071542+0000","last_active":"2023-10-16T08:40:12.071405+0000","last_peered":"2023-10-16T08:40:12.071405+0000","last_clean":"2023-10-16T08:40:12.071405+0000","last_became_active":"2023-10-16T08:38:45.650563+0000","last_became_peered":"2023-10-16T08:38:45.650563+0000","last_unstale":"2023-10-16T08:40:12.071405+0000","last_undegraded":"2023-10-16T08:40:12.071405+0000","last_fullsized":"2023-10-16T08:40:12.071405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:12.071182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:12.071182+0000","last_clean_scrub_stamp":"2023-10-16T08:40:12.071182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:24.423790+0000","scrub_duration":0.0042431159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:56.567995+0000","last_change":"2023-10-16T08:39:56.568099+0000","last_active":"2023-10-16T08:39:56.567995+0000","last_peered":"2023-10-16T08:39:56.567995+0000","last_clean":"2023-10-16T08:39:56.567995+0000","last_became_active":"2023-10-16T08:38:44.662110+0000","last_became_peered":"2023-10-16T08:38:44.662110+0000","last_unstale":"2023-10-16T08:39:56.567995+0000","last_undegraded":"2023-10-16T08:39:56.567995+0000","last_fullsized":"2023-10-16T08:39:56.567995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:56.567779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:56.567779+0000","last_clean_scrub_stamp":"2023-10-16T08:39:56.567779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:37:13.876495+0000","scrub_duration":0.0037362279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:20.235447+0000","last_change":"2023-10-16T08:40:20.235539+0000","last_active":"2023-10-16T08:40:20.235447+0000","last_peered":"2023-10-16T08:40:20.235447+0000","last_clean":"2023-10-16T08:40:20.235447+0000","last_became_active":"2023-10-16T08:38:45.661595+0000","last_became_peered":"2023-10-16T08:38:45.661595+0000","last_unstale":"2023-10-16T08:40:20.235447+0000","last_undegraded":"2023-10-16T08:40:20.235447+0000","last_fullsized":"2023-10-16T08:40:20.235447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:20.235213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:20.235213+0000","last_clean_scrub_stamp":"2023-10-16T08:40:20.235213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:37:01.456143+0000","scrub_duration":0.0035930620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:50.191434+0000","last_change":"2023-10-16T08:39:50.191539+0000","last_active":"2023-10-16T08:39:50.191434+0000","last_peered":"2023-10-16T08:39:50.191434+0000","last_clean":"2023-10-16T08:39:50.191434+0000","last_became_active":"2023-10-16T08:38:44.657153+0000","last_became_peered":"2023-10-16T08:38:44.657153+0000","last_unstale":"2023-10-16T08:39:50.191434+0000","last_undegraded":"2023-10-16T08:39:50.191434+0000","last_fullsized":"2023-10-16T08:39:50.191434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:50.191165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:50.191165+0000","last_clean_scrub_stamp":"2023-10-16T08:39:50.191165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:31:15.292095+0000","scrub_duration":0.0038904819999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:04.049549+0000","last_change":"2023-10-16T08:40:04.049648+0000","last_active":"2023-10-16T08:40:04.049549+0000","last_peered":"2023-10-16T08:40:04.049549+0000","last_clean":"2023-10-16T08:40:04.049549+0000","last_became_active":"2023-10-16T08:38:45.635633+0000","last_became_peered":"2023-10-16T08:38:45.635633+0000","last_unstale":"2023-10-16T08:40:04.049549+0000","last_undegraded":"2023-10-16T08:40:04.049549+0000","last_fullsized":"2023-10-16T08:40:04.049549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:04.049337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:04.049337+0000","last_clean_scrub_stamp":"2023-10-16T08:40:04.049337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:16.401945+0000","scrub_duration":0.0038362539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'12","reported_seq":35,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:46.606020+0000","last_change":"2023-10-16T08:39:46.606097+0000","last_active":"2023-10-16T08:39:46.606020+0000","last_peered":"2023-10-16T08:39:46.606020+0000","last_clean":"2023-10-16T08:39:46.606020+0000","last_became_active":"2023-10-16T08:38:44.637640+0000","last_became_peered":"2023-10-16T08:38:44.637640+0000","last_unstale":"2023-10-16T08:39:46.606020+0000","last_undegraded":"2023-10-16T08:39:46.606020+0000","last_fullsized":"2023-10-16T08:39:46.606020+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'12","last_scrub_stamp":"2023-10-16T08:39:46.605891+0000","last_deep_scrub":"22'12","last_deep_scrub_stamp":"2023-10-16T08:39:46.605891+0000","last_clean_scrub_stamp":"2023-10-16T08:39:46.605891+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:02.027806+0000","scrub_duration":0.0068877740000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:21.278804+0000","last_change":"2023-10-16T08:40:21.278917+0000","last_active":"2023-10-16T08:40:21.278804+0000","last_peered":"2023-10-16T08:40:21.278804+0000","last_clean":"2023-10-16T08:40:21.278804+0000","last_became_active":"2023-10-16T08:38:45.656651+0000","last_became_peered":"2023-10-16T08:38:45.656651+0000","last_unstale":"2023-10-16T08:40:21.278804+0000","last_undegraded":"2023-10-16T08:40:21.278804+0000","last_fullsized":"2023-10-16T08:40:21.278804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:21.278543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:21.278543+0000","last_clean_scrub_stamp":"2023-10-16T08:40:21.278543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:19:12.584357+0000","scrub_duration":0.0037387620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'2","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:47.765260+0000","last_change":"2023-10-16T08:39:47.765365+0000","last_active":"2023-10-16T08:39:47.765260+0000","last_peered":"2023-10-16T08:39:47.765260+0000","last_clean":"2023-10-16T08:39:47.765260+0000","last_became_active":"2023-10-16T08:38:44.647887+0000","last_became_peered":"2023-10-16T08:38:44.647887+0000","last_unstale":"2023-10-16T08:39:47.765260+0000","last_undegraded":"2023-10-16T08:39:47.765260+0000","last_fullsized":"2023-10-16T08:39:47.765260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'2","last_scrub_stamp":"2023-10-16T08:39:47.765049+0000","last_deep_scrub":"22'2","last_deep_scrub_stamp":"2023-10-16T08:39:47.765049+0000","last_clean_scrub_stamp":"2023-10-16T08:39:47.765049+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:03.642054+0000","scrub_duration":0.0054068919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:49.280907+0000","last_change":"2023-10-16T08:39:49.281008+0000","last_active":"2023-10-16T08:39:49.280907+0000","last_peered":"2023-10-16T08:39:49.280907+0000","last_clean":"2023-10-16T08:39:49.280907+0000","last_became_active":"2023-10-16T08:38:45.656750+0000","last_became_peered":"2023-10-16T08:38:45.656750+0000","last_unstale":"2023-10-16T08:39:49.280907+0000","last_undegraded":"2023-10-16T08:39:49.280907+0000","last_fullsized":"2023-10-16T08:39:49.280907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:49.280644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:49.280644+0000","last_clean_scrub_stamp":"2023-10-16T08:39:49.280644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:51:41.348858+0000","scrub_duration":0.0036633019999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'3","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:56.656761+0000","last_change":"2023-10-16T08:39:56.656883+0000","last_active":"2023-10-16T08:39:56.656761+0000","last_peered":"2023-10-16T08:39:56.656761+0000","last_clean":"2023-10-16T08:39:56.656761+0000","last_became_active":"2023-10-16T08:38:44.646042+0000","last_became_peered":"2023-10-16T08:38:44.646042+0000","last_unstale":"2023-10-16T08:39:56.656761+0000","last_undegraded":"2023-10-16T08:39:56.656761+0000","last_fullsized":"2023-10-16T08:39:56.656761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2023-10-16T08:39:56.656531+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2023-10-16T08:39:56.656531+0000","last_clean_scrub_stamp":"2023-10-16T08:39:56.656531+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:31:21.757461+0000","scrub_duration":0.0076556489999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.010439+0000","last_change":"2023-10-16T08:38:45.670969+0000","last_active":"2023-10-16T08:39:27.010439+0000","last_peered":"2023-10-16T08:39:27.010439+0000","last_clean":"2023-10-16T08:39:27.010439+0000","last_became_active":"2023-10-16T08:38:45.670415+0000","last_became_peered":"2023-10-16T08:38:45.670415+0000","last_unstale":"2023-10-16T08:39:27.010439+0000","last_undegraded":"2023-10-16T08:39:27.010439+0000","last_fullsized":"2023-10-16T08:39:27.010439+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'2","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:01.687399+0000","last_change":"2023-10-16T08:40:01.687511+0000","last_active":"2023-10-16T08:40:01.687399+0000","last_peered":"2023-10-16T08:40:01.687399+0000","last_clean":"2023-10-16T08:40:01.687399+0000","last_became_active":"2023-10-16T08:38:44.655260+0000","last_became_peered":"2023-10-16T08:38:44.655260+0000","last_unstale":"2023-10-16T08:40:01.687399+0000","last_undegraded":"2023-10-16T08:40:01.687399+0000","last_fullsized":"2023-10-16T08:40:01.687399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'2","last_scrub_stamp":"2023-10-16T08:40:01.687151+0000","last_deep_scrub":"22'2","last_deep_scrub_stamp":"2023-10-16T08:40:01.687151+0000","last_clean_scrub_stamp":"2023-10-16T08:40:01.687151+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:32:11.992732+0000","scrub_duration":0.0081170519999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:06.489937+0000","last_change":"2023-10-16T08:40:06.490039+0000","last_active":"2023-10-16T08:40:06.489937+0000","last_peered":"2023-10-16T08:40:06.489937+0000","last_clean":"2023-10-16T08:40:06.489937+0000","last_became_active":"2023-10-16T08:38:45.667627+0000","last_became_peered":"2023-10-16T08:38:45.667627+0000","last_unstale":"2023-10-16T08:40:06.489937+0000","last_undegraded":"2023-10-16T08:40:06.489937+0000","last_fullsized":"2023-10-16T08:40:06.489937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:06.489710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:06.489710+0000","last_clean_scrub_stamp":"2023-10-16T08:40:06.489710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:10:38.568870+0000","scrub_duration":0.003795746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:51.623054+0000","last_change":"2023-10-16T08:39:51.623164+0000","last_active":"2023-10-16T08:39:51.623054+0000","last_peered":"2023-10-16T08:39:51.623054+0000","last_clean":"2023-10-16T08:39:51.623054+0000","last_became_active":"2023-10-16T08:38:44.661736+0000","last_became_peered":"2023-10-16T08:38:44.661736+0000","last_unstale":"2023-10-16T08:39:51.623054+0000","last_undegraded":"2023-10-16T08:39:51.623054+0000","last_fullsized":"2023-10-16T08:39:51.623054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:51.622816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:51.622816+0000","last_clean_scrub_stamp":"2023-10-16T08:39:51.622816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:29.535131+0000","scrub_duration":0.0038193530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.009958+0000","last_change":"2023-10-16T08:38:45.667423+0000","last_active":"2023-10-16T08:39:27.009958+0000","last_peered":"2023-10-16T08:39:27.009958+0000","last_clean":"2023-10-16T08:39:27.009958+0000","last_became_active":"2023-10-16T08:38:45.666390+0000","last_became_peered":"2023-10-16T08:38:45.666390+0000","last_unstale":"2023-10-16T08:39:27.009958+0000","last_undegraded":"2023-10-16T08:39:27.009958+0000","last_fullsized":"2023-10-16T08:39:27.009958+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'119","reported_seq":178,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:50.308532+0000","last_change":"2023-10-16T08:39:50.308644+0000","last_active":"2023-10-16T08:39:50.308532+0000","last_peered":"2023-10-16T08:39:50.308532+0000","last_clean":"2023-10-16T08:39:50.308532+0000","last_became_active":"2023-10-16T08:38:34.310944+0000","last_became_peered":"2023-10-16T08:38:34.310944+0000","last_unstale":"2023-10-16T08:39:50.308532+0000","last_undegraded":"2023-10-16T08:39:50.308532+0000","last_fullsized":"2023-10-16T08:39:50.308532+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'119","last_scrub_stamp":"2023-10-16T08:39:50.308327+0000","last_deep_scrub":"22'119","last_deep_scrub_stamp":"2023-10-16T08:39:50.308327+0000","last_clean_scrub_stamp":"2023-10-16T08:39:50.308327+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:14.065843+0000","scrub_duration":0.014557508,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:48.490807+0000","last_change":"2023-10-16T08:39:48.490904+0000","last_active":"2023-10-16T08:39:48.490807+0000","last_peered":"2023-10-16T08:39:48.490807+0000","last_clean":"2023-10-16T08:39:48.490807+0000","last_became_active":"2023-10-16T08:38:44.654277+0000","last_became_peered":"2023-10-16T08:38:44.654277+0000","last_unstale":"2023-10-16T08:39:48.490807+0000","last_undegraded":"2023-10-16T08:39:48.490807+0000","last_fullsized":"2023-10-16T08:39:48.490807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:48.490591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:48.490591+0000","last_clean_scrub_stamp":"2023-10-16T08:39:48.490591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:51:40.558806+0000","scrub_duration":0.0031485229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:16.723373+0000","last_change":"2023-10-16T08:40:16.723476+0000","last_active":"2023-10-16T08:40:16.723373+0000","last_peered":"2023-10-16T08:40:16.723373+0000","last_clean":"2023-10-16T08:40:16.723373+0000","last_became_active":"2023-10-16T08:38:45.679409+0000","last_became_peered":"2023-10-16T08:38:45.679409+0000","last_unstale":"2023-10-16T08:40:16.723373+0000","last_undegraded":"2023-10-16T08:40:16.723373+0000","last_fullsized":"2023-10-16T08:40:16.723373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:16.723108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:16.723108+0000","last_clean_scrub_stamp":"2023-10-16T08:40:16.723108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:46:41.397936+0000","scrub_duration":0.0042941769999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:55.336465+0000","last_change":"2023-10-16T08:39:55.336630+0000","last_active":"2023-10-16T08:39:55.336465+0000","last_peered":"2023-10-16T08:39:55.336465+0000","last_clean":"2023-10-16T08:39:55.336465+0000","last_became_active":"2023-10-16T08:38:44.651534+0000","last_became_peered":"2023-10-16T08:38:44.651534+0000","last_unstale":"2023-10-16T08:39:55.336465+0000","last_undegraded":"2023-10-16T08:39:55.336465+0000","last_fullsized":"2023-10-16T08:39:55.336465+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:55.336229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:55.336229+0000","last_clean_scrub_stamp":"2023-10-16T08:39:55.336229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:37:12.644946+0000","scrub_duration":0.0081414639999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:52.170208+0000","last_change":"2023-10-16T08:39:52.170378+0000","last_active":"2023-10-16T08:39:52.170208+0000","last_peered":"2023-10-16T08:39:52.170208+0000","last_clean":"2023-10-16T08:39:52.170208+0000","last_became_active":"2023-10-16T08:38:45.650715+0000","last_became_peered":"2023-10-16T08:38:45.650715+0000","last_unstale":"2023-10-16T08:39:52.170208+0000","last_undegraded":"2023-10-16T08:39:52.170208+0000","last_fullsized":"2023-10-16T08:39:52.170208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:52.169935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:52.169935+0000","last_clean_scrub_stamp":"2023-10-16T08:39:52.169935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:48:51.865234+0000","scrub_duration":0.0041537630000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:53.623737+0000","last_change":"2023-10-16T08:39:53.623864+0000","last_active":"2023-10-16T08:39:53.623737+0000","last_peered":"2023-10-16T08:39:53.623737+0000","last_clean":"2023-10-16T08:39:53.623737+0000","last_became_active":"2023-10-16T08:38:44.645451+0000","last_became_peered":"2023-10-16T08:38:44.645451+0000","last_unstale":"2023-10-16T08:39:53.623737+0000","last_undegraded":"2023-10-16T08:39:53.623737+0000","last_fullsized":"2023-10-16T08:39:53.623737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:53.623513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:53.623513+0000","last_clean_scrub_stamp":"2023-10-16T08:39:53.623513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:58.932498+0000","scrub_duration":0.0035529649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.008066+0000","last_change":"2023-10-16T08:38:45.641499+0000","last_active":"2023-10-16T08:39:27.008066+0000","last_peered":"2023-10-16T08:39:27.008066+0000","last_clean":"2023-10-16T08:39:27.008066+0000","last_became_active":"2023-10-16T08:38:45.640765+0000","last_became_peered":"2023-10-16T08:38:45.640765+0000","last_unstale":"2023-10-16T08:39:27.008066+0000","last_undegraded":"2023-10-16T08:39:27.008066+0000","last_fullsized":"2023-10-16T08:39:27.008066+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:53.528625+0000","last_change":"2023-10-16T08:39:53.528786+0000","last_active":"2023-10-16T08:39:53.528625+0000","last_peered":"2023-10-16T08:39:53.528625+0000","last_clean":"2023-10-16T08:39:53.528625+0000","last_became_active":"2023-10-16T08:38:44.654622+0000","last_became_peered":"2023-10-16T08:38:44.654622+0000","last_unstale":"2023-10-16T08:39:53.528625+0000","last_undegraded":"2023-10-16T08:39:53.528625+0000","last_fullsized":"2023-10-16T08:39:53.528625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:53.528393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:53.528393+0000","last_clean_scrub_stamp":"2023-10-16T08:39:53.528393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:27.379794+0000","scrub_duration":0.0041697289999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:53.788332+0000","last_change":"2023-10-16T08:39:53.788477+0000","last_active":"2023-10-16T08:39:53.788332+0000","last_peered":"2023-10-16T08:39:53.788332+0000","last_clean":"2023-10-16T08:39:53.788332+0000","last_became_active":"2023-10-16T08:38:45.658594+0000","last_became_peered":"2023-10-16T08:38:45.658594+0000","last_unstale":"2023-10-16T08:39:53.788332+0000","last_undegraded":"2023-10-16T08:39:53.788332+0000","last_fullsized":"2023-10-16T08:39:53.788332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:53.788113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:53.788113+0000","last_clean_scrub_stamp":"2023-10-16T08:39:53.788113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:02.143993+0000","scrub_duration":0.0036952420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:45.752930+0000","last_change":"2023-10-16T08:39:45.752966+0000","last_active":"2023-10-16T08:39:45.752930+0000","last_peered":"2023-10-16T08:39:45.752930+0000","last_clean":"2023-10-16T08:39:45.752930+0000","last_became_active":"2023-10-16T08:38:44.633854+0000","last_became_peered":"2023-10-16T08:38:44.633854+0000","last_unstale":"2023-10-16T08:39:45.752930+0000","last_undegraded":"2023-10-16T08:39:45.752930+0000","last_fullsized":"2023-10-16T08:39:45.752930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:45.752829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:45.752829+0000","last_clean_scrub_stamp":"2023-10-16T08:39:45.752829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:18:59.589626+0000","scrub_duration":0.0015287009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:23.561166+0000","last_change":"2023-10-16T08:40:23.561262+0000","last_active":"2023-10-16T08:40:23.561166+0000","last_peered":"2023-10-16T08:40:23.561166+0000","last_clean":"2023-10-16T08:40:23.561166+0000","last_became_active":"2023-10-16T08:39:27.015266+0000","last_became_peered":"2023-10-16T08:39:27.015266+0000","last_unstale":"2023-10-16T08:40:23.561166+0000","last_undegraded":"2023-10-16T08:40:23.561166+0000","last_fullsized":"2023-10-16T08:40:23.561166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:23.560948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:23.560948+0000","last_clean_scrub_stamp":"2023-10-16T08:40:23.560948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:17.028005+0000","scrub_duration":0.0038406479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:04.332824+0000","last_change":"2023-10-16T08:40:04.332989+0000","last_active":"2023-10-16T08:40:04.332824+0000","last_peered":"2023-10-16T08:40:04.332824+0000","last_clean":"2023-10-16T08:40:04.332824+0000","last_became_active":"2023-10-16T08:38:44.655001+0000","last_became_peered":"2023-10-16T08:38:44.655001+0000","last_unstale":"2023-10-16T08:40:04.332824+0000","last_undegraded":"2023-10-16T08:40:04.332824+0000","last_fullsized":"2023-10-16T08:40:04.332824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:04.332585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:04.332585+0000","last_clean_scrub_stamp":"2023-10-16T08:40:04.332585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:46:29.007413+0000","scrub_duration":0.003821592,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:27.008321+0000","last_change":"2023-10-16T08:38:45.639658+0000","last_active":"2023-10-16T08:39:27.008321+0000","last_peered":"2023-10-16T08:39:27.008321+0000","last_clean":"2023-10-16T08:39:27.008321+0000","last_became_active":"2023-10-16T08:38:45.638702+0000","last_became_peered":"2023-10-16T08:38:45.638702+0000","last_unstale":"2023-10-16T08:39:27.008321+0000","last_undegraded":"2023-10-16T08:39:27.008321+0000","last_fullsized":"2023-10-16T08:39:27.008321+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:38:44.595147+0000","last_clean_scrub_stamp":"2023-10-16T08:38:44.595147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'3","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:06.680541+0000","last_change":"2023-10-16T08:40:06.680647+0000","last_active":"2023-10-16T08:40:06.680541+0000","last_peered":"2023-10-16T08:40:06.680541+0000","last_clean":"2023-10-16T08:40:06.680541+0000","last_became_active":"2023-10-16T08:38:44.648312+0000","last_became_peered":"2023-10-16T08:38:44.648312+0000","last_unstale":"2023-10-16T08:40:06.680541+0000","last_undegraded":"2023-10-16T08:40:06.680541+0000","last_fullsized":"2023-10-16T08:40:06.680541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2023-10-16T08:40:06.680326+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2023-10-16T08:40:06.680326+0000","last_clean_scrub_stamp":"2023-10-16T08:40:06.680326+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:42:24.980087+0000","scrub_duration":0.0047609419999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:27.612182+0000","last_change":"2023-10-16T08:40:27.612282+0000","last_active":"2023-10-16T08:40:27.612182+0000","last_peered":"2023-10-16T08:40:27.612182+0000","last_clean":"2023-10-16T08:40:27.612182+0000","last_became_active":"2023-10-16T08:38:45.662442+0000","last_became_peered":"2023-10-16T08:38:45.662442+0000","last_unstale":"2023-10-16T08:40:27.612182+0000","last_undegraded":"2023-10-16T08:40:27.612182+0000","last_fullsized":"2023-10-16T08:40:27.612182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:27.611937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:27.611937+0000","last_clean_scrub_stamp":"2023-10-16T08:40:27.611937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:24.469054+0000","scrub_duration":0.0040286890000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'3","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:48.214304+0000","last_change":"2023-10-16T08:39:48.214423+0000","last_active":"2023-10-16T08:39:48.214304+0000","last_peered":"2023-10-16T08:39:48.214304+0000","last_clean":"2023-10-16T08:39:48.214304+0000","last_became_active":"2023-10-16T08:38:44.648155+0000","last_became_peered":"2023-10-16T08:38:44.648155+0000","last_unstale":"2023-10-16T08:39:48.214304+0000","last_undegraded":"2023-10-16T08:39:48.214304+0000","last_fullsized":"2023-10-16T08:39:48.214304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2023-10-16T08:39:48.214061+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2023-10-16T08:39:48.214061+0000","last_clean_scrub_stamp":"2023-10-16T08:39:48.214061+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:53.523046+0000","scrub_duration":0.0072707400000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:27.792442+0000","last_change":"2023-10-16T08:40:27.792545+0000","last_active":"2023-10-16T08:40:27.792442+0000","last_peered":"2023-10-16T08:40:27.792442+0000","last_clean":"2023-10-16T08:40:27.792442+0000","last_became_active":"2023-10-16T08:38:45.636751+0000","last_became_peered":"2023-10-16T08:38:45.636751+0000","last_unstale":"2023-10-16T08:40:27.792442+0000","last_undegraded":"2023-10-16T08:40:27.792442+0000","last_fullsized":"2023-10-16T08:40:27.792442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:27.792223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:27.792223+0000","last_clean_scrub_stamp":"2023-10-16T08:40:27.792223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:07:51.515524+0000","scrub_duration":0.0034292670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:51.303765+0000","last_change":"2023-10-16T08:39:51.303866+0000","last_active":"2023-10-16T08:39:51.303765+0000","last_peered":"2023-10-16T08:39:51.303765+0000","last_clean":"2023-10-16T08:39:51.303765+0000","last_became_active":"2023-10-16T08:38:44.646990+0000","last_became_peered":"2023-10-16T08:38:44.646990+0000","last_unstale":"2023-10-16T08:39:51.303765+0000","last_undegraded":"2023-10-16T08:39:51.303765+0000","last_fullsized":"2023-10-16T08:39:51.303765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:51.303518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:51.303518+0000","last_clean_scrub_stamp":"2023-10-16T08:39:51.303518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:55:30.823021+0000","scrub_duration":0.0039969649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:48.807872+0000","last_change":"2023-10-16T08:39:48.807982+0000","last_active":"2023-10-16T08:39:48.807872+0000","last_peered":"2023-10-16T08:39:48.807872+0000","last_clean":"2023-10-16T08:39:48.807872+0000","last_became_active":"2023-10-16T08:38:45.647449+0000","last_became_peered":"2023-10-16T08:38:45.647449+0000","last_unstale":"2023-10-16T08:39:48.807872+0000","last_undegraded":"2023-10-16T08:39:48.807872+0000","last_fullsized":"2023-10-16T08:39:48.807872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:48.807616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:48.807616+0000","last_clean_scrub_stamp":"2023-10-16T08:39:48.807616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:04.229531+0000","scrub_duration":0.0038025490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:48.714882+0000","last_change":"2023-10-16T08:39:48.714951+0000","last_active":"2023-10-16T08:39:48.714882+0000","last_peered":"2023-10-16T08:39:48.714882+0000","last_clean":"2023-10-16T08:39:48.714882+0000","last_became_active":"2023-10-16T08:38:44.630451+0000","last_became_peered":"2023-10-16T08:38:44.630451+0000","last_unstale":"2023-10-16T08:39:48.714882+0000","last_undegraded":"2023-10-16T08:39:48.714882+0000","last_fullsized":"2023-10-16T08:39:48.714882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:48.714761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:48.714761+0000","last_clean_scrub_stamp":"2023-10-16T08:39:48.714761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:42:57.070641+0000","scrub_duration":0.0033819409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:13.680916+0000","last_change":"2023-10-16T08:40:13.681012+0000","last_active":"2023-10-16T08:40:13.680916+0000","last_peered":"2023-10-16T08:40:13.680916+0000","last_clean":"2023-10-16T08:40:13.680916+0000","last_became_active":"2023-10-16T08:38:45.663100+0000","last_became_peered":"2023-10-16T08:38:45.663100+0000","last_unstale":"2023-10-16T08:40:13.680916+0000","last_undegraded":"2023-10-16T08:40:13.680916+0000","last_fullsized":"2023-10-16T08:40:13.680916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:13.680679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:13.680679+0000","last_clean_scrub_stamp":"2023-10-16T08:40:13.680679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:24.518729+0000","scrub_duration":0.0037025669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:10.638658+0000","last_change":"2023-10-16T08:40:10.638842+0000","last_active":"2023-10-16T08:40:10.638658+0000","last_peered":"2023-10-16T08:40:10.638658+0000","last_clean":"2023-10-16T08:40:10.638658+0000","last_became_active":"2023-10-16T08:38:44.646580+0000","last_became_peered":"2023-10-16T08:38:44.646580+0000","last_unstale":"2023-10-16T08:40:10.638658+0000","last_undegraded":"2023-10-16T08:40:10.638658+0000","last_fullsized":"2023-10-16T08:40:10.638658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:10.638415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:10.638415+0000","last_clean_scrub_stamp":"2023-10-16T08:40:10.638415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:50:05.087112+0000","scrub_duration":0.003785058,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:12.671707+0000","last_change":"2023-10-16T08:40:12.671827+0000","last_active":"2023-10-16T08:40:12.671707+0000","last_peered":"2023-10-16T08:40:12.671707+0000","last_clean":"2023-10-16T08:40:12.671707+0000","last_became_active":"2023-10-16T08:38:45.662226+0000","last_became_peered":"2023-10-16T08:38:45.662226+0000","last_unstale":"2023-10-16T08:40:12.671707+0000","last_undegraded":"2023-10-16T08:40:12.671707+0000","last_fullsized":"2023-10-16T08:40:12.671707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:12.671469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:12.671469+0000","last_clean_scrub_stamp":"2023-10-16T08:40:12.671469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:13:45.857171+0000","scrub_duration":0.003468933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'3","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:53.387291+0000","last_change":"2023-10-16T08:39:53.387482+0000","last_active":"2023-10-16T08:39:53.387291+0000","last_peered":"2023-10-16T08:39:53.387291+0000","last_clean":"2023-10-16T08:39:53.387291+0000","last_became_active":"2023-10-16T08:38:44.659976+0000","last_became_peered":"2023-10-16T08:38:44.659976+0000","last_unstale":"2023-10-16T08:39:53.387291+0000","last_undegraded":"2023-10-16T08:39:53.387291+0000","last_fullsized":"2023-10-16T08:39:53.387291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2023-10-16T08:39:53.387049+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2023-10-16T08:39:53.387049+0000","last_clean_scrub_stamp":"2023-10-16T08:39:53.387049+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:27.238450+0000","scrub_duration":0.0071616939999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:08.969126+0000","last_change":"2023-10-16T08:40:08.969225+0000","last_active":"2023-10-16T08:40:08.969126+0000","last_peered":"2023-10-16T08:40:08.969126+0000","last_clean":"2023-10-16T08:40:08.969126+0000","last_became_active":"2023-10-16T08:38:45.664616+0000","last_became_peered":"2023-10-16T08:38:45.664616+0000","last_unstale":"2023-10-16T08:40:08.969126+0000","last_undegraded":"2023-10-16T08:40:08.969126+0000","last_fullsized":"2023-10-16T08:40:08.969126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:08.968899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:08.968899+0000","last_clean_scrub_stamp":"2023-10-16T08:40:08.968899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:50:24.351118+0000","scrub_duration":0.0033892219999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:13.566725+0000","last_change":"2023-10-16T08:40:13.566817+0000","last_active":"2023-10-16T08:40:13.566725+0000","last_peered":"2023-10-16T08:40:13.566725+0000","last_clean":"2023-10-16T08:40:13.566725+0000","last_became_active":"2023-10-16T08:38:45.680231+0000","last_became_peered":"2023-10-16T08:38:45.680231+0000","last_unstale":"2023-10-16T08:40:13.566725+0000","last_undegraded":"2023-10-16T08:40:13.566725+0000","last_fullsized":"2023-10-16T08:40:13.566725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:13.566499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:13.566499+0000","last_clean_scrub_stamp":"2023-10-16T08:40:13.566499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:36:54.787429+0000","scrub_duration":0.0033674909999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'3","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:58.403119+0000","last_change":"2023-10-16T08:39:58.403247+0000","last_active":"2023-10-16T08:39:58.403119+0000","last_peered":"2023-10-16T08:39:58.403119+0000","last_clean":"2023-10-16T08:39:58.403119+0000","last_became_active":"2023-10-16T08:38:44.663674+0000","last_became_peered":"2023-10-16T08:38:44.663674+0000","last_unstale":"2023-10-16T08:39:58.403119+0000","last_undegraded":"2023-10-16T08:39:58.403119+0000","last_fullsized":"2023-10-16T08:39:58.403119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2023-10-16T08:39:58.402923+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2023-10-16T08:39:58.402923+0000","last_clean_scrub_stamp":"2023-10-16T08:39:58.402923+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:13:31.588625+0000","scrub_duration":0.0069404790000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:08.562697+0000","last_change":"2023-10-16T08:40:08.562797+0000","last_active":"2023-10-16T08:40:08.562697+0000","last_peered":"2023-10-16T08:40:08.562697+0000","last_clean":"2023-10-16T08:40:08.562697+0000","last_became_active":"2023-10-16T08:38:45.680266+0000","last_became_peered":"2023-10-16T08:38:45.680266+0000","last_unstale":"2023-10-16T08:40:08.562697+0000","last_undegraded":"2023-10-16T08:40:08.562697+0000","last_fullsized":"2023-10-16T08:40:08.562697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:08.562471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:08.562471+0000","last_clean_scrub_stamp":"2023-10-16T08:40:08.562471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:29:33.941649+0000","scrub_duration":0.0036526010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'3","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:01.344970+0000","last_change":"2023-10-16T08:40:01.345161+0000","last_active":"2023-10-16T08:40:01.344970+0000","last_peered":"2023-10-16T08:40:01.344970+0000","last_clean":"2023-10-16T08:40:01.344970+0000","last_became_active":"2023-10-16T08:38:44.652658+0000","last_became_peered":"2023-10-16T08:38:44.652658+0000","last_unstale":"2023-10-16T08:40:01.344970+0000","last_undegraded":"2023-10-16T08:40:01.344970+0000","last_fullsized":"2023-10-16T08:40:01.344970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2023-10-16T08:40:01.344742+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2023-10-16T08:40:01.344742+0000","last_clean_scrub_stamp":"2023-10-16T08:40:01.344742+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:12.182792+0000","scrub_duration":0.0070571119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:24.819126+0000","last_change":"2023-10-16T08:40:24.819221+0000","last_active":"2023-10-16T08:40:24.819126+0000","last_peered":"2023-10-16T08:40:24.819126+0000","last_clean":"2023-10-16T08:40:24.819126+0000","last_became_active":"2023-10-16T08:38:45.664274+0000","last_became_peered":"2023-10-16T08:38:45.664274+0000","last_unstale":"2023-10-16T08:40:24.819126+0000","last_undegraded":"2023-10-16T08:40:24.819126+0000","last_fullsized":"2023-10-16T08:40:24.819126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:24.818872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:24.818872+0000","last_clean_scrub_stamp":"2023-10-16T08:40:24.818872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:13:58.004574+0000","scrub_duration":0.0037996509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:56.005164+0000","last_change":"2023-10-16T08:39:56.005280+0000","last_active":"2023-10-16T08:39:56.005164+0000","last_peered":"2023-10-16T08:39:56.005164+0000","last_clean":"2023-10-16T08:39:56.005164+0000","last_became_active":"2023-10-16T08:38:44.645000+0000","last_became_peered":"2023-10-16T08:38:44.645000+0000","last_unstale":"2023-10-16T08:39:56.005164+0000","last_undegraded":"2023-10-16T08:39:56.005164+0000","last_fullsized":"2023-10-16T08:39:56.005164+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"19'1","last_scrub_stamp":"2023-10-16T08:39:56.004937+0000","last_deep_scrub":"19'1","last_deep_scrub_stamp":"2023-10-16T08:39:56.004937+0000","last_clean_scrub_stamp":"2023-10-16T08:39:56.004937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:42:14.304698+0000","scrub_duration":0.0037060180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:20.647499+0000","last_change":"2023-10-16T08:40:20.647673+0000","last_active":"2023-10-16T08:40:20.647499+0000","last_peered":"2023-10-16T08:40:20.647499+0000","last_clean":"2023-10-16T08:40:20.647499+0000","last_became_active":"2023-10-16T08:38:45.663538+0000","last_became_peered":"2023-10-16T08:38:45.663538+0000","last_unstale":"2023-10-16T08:40:20.647499+0000","last_undegraded":"2023-10-16T08:40:20.647499+0000","last_fullsized":"2023-10-16T08:40:20.647499+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:20.647257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:20.647257+0000","last_clean_scrub_stamp":"2023-10-16T08:40:20.647257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:30:20.728617+0000","scrub_duration":0.0041945580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'6","reported_seq":29,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:47.192216+0000","last_change":"2023-10-16T08:39:47.192267+0000","last_active":"2023-10-16T08:39:47.192216+0000","last_peered":"2023-10-16T08:39:47.192216+0000","last_clean":"2023-10-16T08:39:47.192216+0000","last_became_active":"2023-10-16T08:38:44.645854+0000","last_became_peered":"2023-10-16T08:38:44.645854+0000","last_unstale":"2023-10-16T08:39:47.192216+0000","last_undegraded":"2023-10-16T08:39:47.192216+0000","last_fullsized":"2023-10-16T08:39:47.192216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'6","last_scrub_stamp":"2023-10-16T08:39:47.192138+0000","last_deep_scrub":"22'6","last_deep_scrub_stamp":"2023-10-16T08:39:47.192138+0000","last_clean_scrub_stamp":"2023-10-16T08:39:47.192138+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:31:12.293068+0000","scrub_duration":0.0072348439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6065,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":12130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":827392,"data_stored":674262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.1919999999999999,"5min":1.1919999999999999,"15min":1.1919999999999999},"last":1.0669999999999999},{"interface":"front","average":{"1min":1.002,"5min":1.002,"15min":1.002},"min":{"1min":0.626,"5min":0.626,"15min":0.626},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":1.244}]},{"osd":1,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.91600000000000004}]},{"osd":2,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.0029999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"min":{"1min":0.62,"5min":0.62,"15min":0.62},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":0.875}]},{"osd":3,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.86099999999999999,"5min":0.86099999999999999,"15min":0.86099999999999999},"min":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":1.036},{"interface":"front","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.746,"5min":0.746,"15min":0.746},"max":{"1min":1.4099999999999999,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":1.1040000000000001}]},{"osd":4,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.874,"5min":0.874,"15min":0.874},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"max":{"1min":1.224,"5min":1.224,"15min":1.224},"last":0.96599999999999997}]},{"osd":5,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.127,"5min":1.127,"15min":1.127},"min":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"max":{"1min":1.3600000000000001,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":1.012},{"interface":"front","average":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.0569999999999999},"min":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"max":{"1min":1.304,"5min":1.304,"15min":1.304},"last":1.21}]},{"osd":6,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":1.141},{"interface":"front","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.747,"5min":0.747,"15min":0.747},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":1.173}]}]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":88579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.1060000000000001},{"interface":"front","average":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.1899999999999999,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.0660000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.98599999999999999},{"interface":"front","average":{"1min":1.018,"5min":1.018,"15min":1.018},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.282,"5min":2.282,"15min":2.282},"last":1.153}]},{"osd":2,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.4530000000000001,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":1.1879999999999999},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.84899999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"min":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.784,"5min":1.784,"15min":1.784},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.999,"5min":0.999,"15min":0.999},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.64400000000000002}]},{"osd":4,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"min":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"max":{"1min":1.5600000000000001,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.75},{"interface":"front","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"max":{"1min":1.925,"5min":1.925,"15min":1.925},"last":0.68700000000000006}]},{"osd":5,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.01,"5min":1.01,"15min":1.01},"min":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.6240000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.90300000000000002},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":2.1150000000000002,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":0.94699999999999995}]},{"osd":7,"last update":"Mon Oct 16 08:39:32 2023","interfaces":[{"interface":"back","average":{"1min":1.2470000000000001,"5min":1.2470000000000001,"15min":1.2470000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":2.04,"5min":2.04,"15min":2.04},"last":1.248},{"interface":"front","average":{"1min":1.1539999999999999,"5min":1.1539999999999999,"15min":1.1539999999999999},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":1.675,"5min":1.675,"15min":1.675},"last":1.02}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":83402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.161,"5min":1.161,"15min":1.161},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.5149999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.871}]},{"osd":1,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.3149999999999999,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":1.1539999999999999},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"min":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"max":{"1min":1.27,"5min":1.27,"15min":1.27},"last":0.92400000000000004}]},{"osd":2,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":1.014},{"interface":"front","average":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.5569999999999999,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":1.048}]},{"osd":3,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"min":{"1min":0.70099999999999996,"5min":0.70099999999999996,"15min":0.70099999999999996},"max":{"1min":1.1559999999999999,"5min":1.1559999999999999,"15min":1.1559999999999999},"last":1.0800000000000001},{"interface":"front","average":{"1min":1.032,"5min":1.032,"15min":1.032},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":2.0209999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.1970000000000001}]},{"osd":4,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.94299999999999995,"5min":0.94299999999999995,"15min":0.94299999999999995},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.696,"5min":1.696,"15min":1.696},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"min":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.66300000000000003}]},{"osd":6,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.079,"5min":1.079,"15min":1.079},"min":{"1min":0.621,"5min":0.621,"15min":0.621},"max":{"1min":2.1040000000000001,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.63900000000000001,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":2.2200000000000002,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":0.82099999999999995}]},{"osd":7,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.496,"5min":1.496,"15min":1.496},"last":0.97499999999999998},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"min":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"max":{"1min":2.181,"5min":2.181,"15min":2.181},"last":1.1160000000000001}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":241664,"data_stored":83438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95299999999999996},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.583,"5min":1.583,"15min":1.583},"last":0.73199999999999998},{"interface":"front","average":{"1min":0.92400000000000004,"5min":0.92400000000000004,"15min":0.92400000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.5900000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":1.0600000000000001}]},{"osd":1,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.716,"5min":1.716,"15min":1.716},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.5049999999999999,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":1.103}]},{"osd":2,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"max":{"1min":1.627,"5min":1.627,"15min":1.627},"last":0.78900000000000003},{"interface":"front","average":{"1min":1.012,"5min":1.012,"15min":1.012},"min":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.476,"5min":1.476,"15min":1.476},"last":0.83499999999999996}]},{"osd":3,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"min":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.0740000000000001},"min":{"1min":0.63300000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.872}]},{"osd":5,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.91300000000000003},{"interface":"front","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.623,"5min":1.623,"15min":1.623},"last":1.1759999999999999}]},{"osd":6,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.742,"5min":1.742,"15min":1.742},"last":1.139},{"interface":"front","average":{"1min":1.157,"5min":1.157,"15min":1.157},"min":{"1min":0.60599999999999998,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.657,"5min":1.657,"15min":1.657},"last":0.97399999999999998}]},{"osd":7,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.069,"15min":1.069},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.667,"5min":1.667,"15min":1.667},"last":1.258},{"interface":"front","average":{"1min":1.22,"5min":1.22,"15min":1.22},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":2.153,"5min":2.153,"15min":2.153},"last":1.012}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":88579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.626},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":1.1879999999999999}]},{"osd":1,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":1.284,"5min":1.284,"15min":1.284},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":1.016}]},{"osd":2,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.337,"5min":1.337,"15min":1.337},"last":0.86599999999999999}]},{"osd":4,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"min":{"1min":0.57799999999999996,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.177,"5min":1.177,"15min":1.177},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"min":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.395,"5min":1.395,"15min":1.395},"last":1.0629999999999999}]},{"osd":5,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.008,"5min":1.008,"15min":1.008},"min":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.5489999999999999,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":0.91700000000000004},{"interface":"front","average":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":1.296}]},{"osd":6,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0149999999999999,"5min":1.0149999999999999,"15min":1.0149999999999999},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.6619999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.73,"5min":1.73,"15min":1.73},"last":1.1559999999999999}]},{"osd":7,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.609,"5min":1.609,"15min":1.609},"last":1.224},{"interface":"front","average":{"1min":1.0920000000000001,"5min":1.0920000000000001,"15min":1.0920000000000001},"min":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.837,"5min":1.837,"15min":1.837},"last":1.115}]}]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":83312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.249,"5min":0.249,"15min":0.249},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.439},{"interface":"front","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.272,"5min":1.272,"15min":1.272},"last":0.71299999999999997}]},{"osd":1,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":1.0960000000000001}]},{"osd":3,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.995,"5min":0.995,"15min":0.995},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.9810000000000001,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":0.86699999999999999}]},{"osd":4,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"min":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.6180000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.0129999999999999},{"interface":"front","average":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"min":{"1min":0.627,"5min":0.627,"15min":0.627},"max":{"1min":1.343,"5min":1.343,"15min":1.343},"last":0.82499999999999996}]},{"osd":5,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.92600000000000005},{"interface":"front","average":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"min":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":2.0670000000000002,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.1419999999999999}]},{"osd":6,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0349999999999999,"5min":1.0349999999999999,"15min":1.0349999999999999},"min":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"max":{"1min":1.3979999999999999,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.2070000000000001},{"interface":"front","average":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"min":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"max":{"1min":1.5,"5min":1.5,"15min":1.5},"last":0.98199999999999998}]},{"osd":7,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"min":{"1min":0.622,"5min":0.622,"15min":0.622},"max":{"1min":1.544,"5min":1.544,"15min":1.544},"last":1.3320000000000001},{"interface":"front","average":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.748,"5min":1.748,"15min":1.748},"last":1.1799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":83472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.81299999999999994}]},{"osd":2,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.76300000000000001,"15min":0.76300000000000001},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.73999999999999999}]},{"osd":3,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.794,"5min":1.794,"15min":1.794},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.749,"5min":0.749,"15min":0.749},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.3140000000000001,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":0.90400000000000003}]},{"osd":4,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.02,"5min":1.02,"15min":1.02},"min":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":2.27,"5min":2.27,"15min":2.27},"last":1.1160000000000001},{"interface":"front","average":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.4510000000000001,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.86799999999999999}]},{"osd":5,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.8919999999999999,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":1.012},{"interface":"front","average":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"min":{"1min":0.55900000000000005,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.54,"5min":1.54,"15min":1.54},"last":1.0800000000000001}]},{"osd":6,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.6830000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.3280000000000001},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.994,"5min":1.994,"15min":1.994},"last":0.93000000000000005}]},{"osd":7,"last update":"Mon Oct 16 08:39:33 2023","interfaces":[{"interface":"back","average":{"1min":1.139,"5min":1.139,"15min":1.139},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":2.1930000000000001,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.1479999999999999},{"interface":"front","average":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"min":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":2.5449999999999999,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":1.2749999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":835584,"data_stored":674318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":1.0509999999999999,"5min":1.0509999999999999,"15min":1.0509999999999999},"last":0.80000000000000004}]},{"osd":2,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":1.421,"5min":1.421,"15min":1.421},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.2769999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":0.89600000000000002}]},{"osd":3,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98099999999999998,"5min":0.98099999999999998,"15min":0.98099999999999998},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.246,"5min":1.246,"15min":1.246},"last":1.075}]},{"osd":4,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":1.0249999999999999},{"interface":"front","average":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.3540000000000001,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.752}]},{"osd":5,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":1.4330000000000001},{"interface":"front","average":{"1min":1.006,"5min":1.006,"15min":1.006},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.522,"5min":1.522,"15min":1.522},"last":1.1539999999999999}]},{"osd":6,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":0.99099999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.4690000000000001,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.1870000000000001},{"interface":"front","average":{"1min":1.044,"5min":1.044,"15min":1.044},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.4490000000000001,"5min":1.4490000000000001,"15min":1.4490000000000001},"last":1.488}]},{"osd":7,"last update":"Mon Oct 16 08:39:34 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":1.014,"15min":1.014},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.4910000000000001,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.345},{"interface":"front","average":{"1min":1.073,"5min":1.073,"15min":1.073},"min":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":1.2250000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:40:29.503 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:29.503 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:29.504 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:29.504 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:29.505 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:29.505 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:29.506 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:29.506 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:29.507 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:29.507 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2023-10-16T08:38:44.595147+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=38, tm_sec=44, tm_wday=0, tm_yday=289, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=16, tm_hour=8, tm_min=39, tm_sec=43, tm_wday=0, tm_yday=289, tm_isdst=0) 2023-10-16T08:40:29.508 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-16T08:40:43.451 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:40:43.454 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:40:49.509 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-16T08:40:49.643 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2c1d58640 1 Processor -- start 2023-10-16T08:40:49.644 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2c1d58640 1 -- start start 2023-10-16T08:40:49.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2c1d58640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fb2bc0cddc0 0x7fb2bc0d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:49.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2c1d58640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2bc0b7f80 con 0x7fb2bc0cd960 2023-10-16T08:40:49.647 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2c1d58640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2bc0b7370 con 0x7fb2bc15cce0 2023-10-16T08:40:49.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2c1d58640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2bc0b7650 con 0x7fb2bc0cddc0 2023-10-16T08:40:49.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2b2ffd640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fb2bc0cd960 0x7fb2bc0d0d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46258/0 (socket says 172.21.15.40:46258) 2023-10-16T08:40:49.648 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2b2ffd640 1 -- 172.21.15.40:0/3375485828 learned_addr learned my addr 172.21.15.40:0/3375485828 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:40:49.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2bbfff640 1 --2- 172.21.15.40:0/3375485828 >> v2:172.21.15.86:3300/0 conn(0x7fb2bc0cddc0 0x7fb2bc0d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:49.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2baffd640 1 -- 172.21.15.40:0/3375485828 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482608974 0 0) 0x7fb2bc0b7f80 con 0x7fb2bc0cd960 2023-10-16T08:40:49.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2baffd640 1 -- 172.21.15.40:0/3375485828 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2a8003530 con 0x7fb2bc0cd960 2023-10-16T08:40:49.649 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2baffd640 1 -- 172.21.15.40:0/3375485828 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1548113631 0 0) 0x7fb2a8003530 con 0x7fb2bc0cd960 2023-10-16T08:40:49.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2baffd640 1 -- 172.21.15.40:0/3375485828 >> v2:172.21.15.86:3300/0 conn(0x7fb2bc0cddc0 msgr2=0x7fb2bc0d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:40:49.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2baffd640 1 --2- 172.21.15.40:0/3375485828 >> v2:172.21.15.86:3300/0 conn(0x7fb2bc0cddc0 0x7fb2bc0d1450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:49.650 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2baffd640 1 -- 172.21.15.40:0/3375485828 >> v1:172.21.15.86:6789/0 conn(0x7fb2bc15cce0 legacy=0x7fb2bc0b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:49.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2baffd640 1 -- 172.21.15.40:0/3375485828 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc0b7930 con 0x7fb2bc0cd960 2023-10-16T08:40:49.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2baffd640 1 -- 172.21.15.40:0/3375485828 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7fb2a4001680 con 0x7fb2bc0cd960 2023-10-16T08:40:49.651 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2baffd640 1 -- 172.21.15.40:0/3375485828 >> v1:172.21.15.40:6789/0 conn(0x7fb2bc0cd960 legacy=0x7fb2bc0d0d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:49.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2baffd640 1 --2- 172.21.15.40:0/3375485828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2a8003a40 0x7fb2a8003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:49.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2baffd640 1 --2- 172.21.15.40:0/3375485828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a80047d0 0x7fb2a8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:49.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2baffd640 1 --2- 172.21.15.40:0/3375485828 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2a80051e0 0x7fb2a800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:49.652 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2baffd640 1 -- 172.21.15.40:0/3375485828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0b7650 con 0x7fb2a80047d0 2023-10-16T08:40:49.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2baffd640 1 -- 172.21.15.40:0/3375485828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a800c110 con 0x7fb2a8003a40 2023-10-16T08:40:49.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2baffd640 1 -- 172.21.15.40:0/3375485828 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2a800c2d0 con 0x7fb2a80051e0 2023-10-16T08:40:49.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2b2ffd640 1 --2- 172.21.15.40:0/3375485828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a80047d0 0x7fb2a8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:49.653 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2bb7fe640 1 --2- 172.21.15.40:0/3375485828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2a8003a40 0x7fb2a8003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:49.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2bbfff640 1 --2- 172.21.15.40:0/3375485828 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2a80051e0 0x7fb2a800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:49.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2b2ffd640 1 -- 172.21.15.40:0/3375485828 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2a80051e0 msgr2=0x7fb2a800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:40:49.654 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2b2ffd640 1 --2- 172.21.15.40:0/3375485828 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2a80051e0 0x7fb2a800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:49.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2b2ffd640 1 -- 172.21.15.40:0/3375485828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2a8003a40 msgr2=0x7fb2a8003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:40:49.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2b2ffd640 1 --2- 172.21.15.40:0/3375485828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2a8003a40 0x7fb2a8003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:49.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2b2ffd640 1 -- 172.21.15.40:0/3375485828 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2a8003530 con 0x7fb2a80047d0 2023-10-16T08:40:49.655 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2bbfff640 1 --2- 172.21.15.40:0/3375485828 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2a80051e0 0x7fb2a800b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:40:49.656 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.641+0000 7fb2bbfff640 1 -- 172.21.15.40:0/3375485828 reap_dead start 2023-10-16T08:40:49.659 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.645+0000 7fb2b2ffd640 1 --2- 172.21.15.40:0/3375485828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a80047d0 0x7fb2a8004c50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb2a400c0e0 tx=0x7fb2a401f250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4f0932be8dcd582 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:49.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.645+0000 7fb2baffd640 1 -- 172.21.15.40:0/3375485828 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a40a8e70 con 0x7fb2a80047d0 2023-10-16T08:40:49.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.645+0000 7fb2baffd640 1 -- 172.21.15.40:0/3375485828 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2a40058e0 con 0x7fb2a80047d0 2023-10-16T08:40:49.660 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.645+0000 7fb2baffd640 1 -- 172.21.15.40:0/3375485828 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2a4005d10 con 0x7fb2a80047d0 2023-10-16T08:40:49.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.645+0000 7fb2c1d58640 1 -- 172.21.15.40:0/3375485828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a80047d0 msgr2=0x7fb2a8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:49.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.645+0000 7fb2c1d58640 1 --2- 172.21.15.40:0/3375485828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a80047d0 0x7fb2a8004c50 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb2a400c0e0 tx=0x7fb2a401f250 comp rx=0 tx=0).stop 2023-10-16T08:40:49.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.645+0000 7fb2c1d58640 1 -- 172.21.15.40:0/3375485828 shutdown_connections 2023-10-16T08:40:49.661 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.645+0000 7fb2c1d58640 1 --2- 172.21.15.40:0/3375485828 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a80047d0 0x7fb2a8004c50 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:49.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.645+0000 7fb2c1d58640 1 -- 172.21.15.40:0/3375485828 >> 172.21.15.40:0/3375485828 conn(0x7fb2bc06a290 msgr2=0x7fb2bc06a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:49.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.645+0000 7fb2c1d58640 1 -- 172.21.15.40:0/3375485828 shutdown_connections 2023-10-16T08:40:49.662 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.645+0000 7fb2c1d58640 1 -- 172.21.15.40:0/3375485828 wait complete. 2023-10-16T08:40:49.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.645+0000 7fb2c1d58640 1 Processor -- start 2023-10-16T08:40:49.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.645+0000 7fb2c1d58640 1 -- start start 2023-10-16T08:40:49.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2c1d58640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a80047d0 0x7fb2bc152d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:49.663 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2c1d58640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2bc14f0f0 0x7fb2bc0c9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:49.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2c1d58640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2bc0cad60 0x7fb2bc0cb210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:49.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2c1d58640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0cb750 con 0x7fb2a80047d0 2023-10-16T08:40:49.664 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2c1d58640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc151a30 con 0x7fb2bc0cad60 2023-10-16T08:40:49.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2c1d58640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2bc0ca2a0 con 0x7fb2bc14f0f0 2023-10-16T08:40:49.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2bb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a80047d0 0x7fb2bc152d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:49.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2bb7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a80047d0 0x7fb2bc152d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39832/0 (socket says 172.21.15.40:39832) 2023-10-16T08:40:49.665 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2bb7fe640 1 -- 172.21.15.40:0/637005239 learned_addr learned my addr 172.21.15.40:0/637005239 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:40:49.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2bb7fe640 1 -- 172.21.15.40:0/637005239 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2bc14f0f0 msgr2=0x7fb2bc0c9d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:49.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2bb7fe640 1 --2- 172.21.15.40:0/637005239 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2bc14f0f0 0x7fb2bc0c9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:49.666 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2bb7fe640 1 -- 172.21.15.40:0/637005239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2bc0cad60 msgr2=0x7fb2bc0cb210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:49.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2bbfff640 1 --2- 172.21.15.40:0/637005239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2bc0cad60 0x7fb2bc0cb210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:49.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2bb7fe640 1 --2- 172.21.15.40:0/637005239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2bc0cad60 0x7fb2bc0cb210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:49.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2bb7fe640 1 -- 172.21.15.40:0/637005239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2bc151c80 con 0x7fb2a80047d0 2023-10-16T08:40:49.667 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2bb7fe640 1 --2- 172.21.15.40:0/637005239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a80047d0 0x7fb2bc152d20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00e4b0 tx=0x7fb2ac00e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:49.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2b8ff9640 1 -- 172.21.15.40:0/637005239 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac0052e0 con 0x7fb2a80047d0 2023-10-16T08:40:49.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2b8ff9640 1 -- 172.21.15.40:0/637005239 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ac005440 con 0x7fb2a80047d0 2023-10-16T08:40:49.668 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2b8ff9640 1 -- 172.21.15.40:0/637005239 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ac003800 con 0x7fb2a80047d0 2023-10-16T08:40:49.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2c1d58640 1 -- 172.21.15.40:0/637005239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2bc151f60 con 0x7fb2a80047d0 2023-10-16T08:40:49.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2c1d58640 1 -- 172.21.15.40:0/637005239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2bc152520 con 0x7fb2a80047d0 2023-10-16T08:40:49.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2c1d58640 1 -- 172.21.15.40:0/637005239 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2bc04e9c0 con 0x7fb2a80047d0 2023-10-16T08:40:49.669 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2b8ff9640 1 -- 172.21.15.40:0/637005239 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fb2ac003960 con 0x7fb2a80047d0 2023-10-16T08:40:49.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2b8ff9640 1 --2- 172.21.15.40:0/637005239 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb29c0664d0 0x7fb29c068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:49.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2b8ff9640 1 -- 172.21.15.40:0/637005239 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fb2ac083c70 con 0x7fb2a80047d0 2023-10-16T08:40:49.670 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.649+0000 7fb2b2ffd640 1 --2- 172.21.15.40:0/637005239 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb29c0664d0 0x7fb29c068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.653+0000 7fb2b2ffd640 1 --2- 172.21.15.40:0/637005239 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb29c0664d0 0x7fb29c068990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40ae000 tx=0x7fb2a4001940 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:49.671 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.653+0000 7fb2b8ff9640 1 -- 172.21.15.40:0/637005239 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fb2ac051040 con 0x7fb2a80047d0 2023-10-16T08:40:49.812 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.809+0000 7fb2c1d58640 1 -- 172.21.15.40:0/637005239 --> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb2bc0ce250 con 0x7fb29c0664d0 2023-10-16T08:40:49.835 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.829+0000 7fb2b8ff9640 1 -- 172.21.15.40:0/637005239 <== mgr.4111 v2:172.21.15.86:6832/1188970899 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288555 (secure 0 0 0) 0x7fb2a4001480 con 0x7fb29c0664d0 2023-10-16T08:40:49.838 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:40:49.840 INFO:teuthology.orchestra.run.smithi040.stderr:dumped all 2023-10-16T08:40:49.841 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.833+0000 7fb2c1d58640 1 -- 172.21.15.40:0/637005239 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb29c0664d0 msgr2=0x7fb29c068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:49.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.833+0000 7fb2c1d58640 1 --2- 172.21.15.40:0/637005239 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb29c0664d0 0x7fb29c068990 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb2a40ae000 tx=0x7fb2a4001940 comp rx=0 tx=0).stop 2023-10-16T08:40:49.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.833+0000 7fb2c1d58640 1 -- 172.21.15.40:0/637005239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a80047d0 msgr2=0x7fb2bc152d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:49.842 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.833+0000 7fb2c1d58640 1 --2- 172.21.15.40:0/637005239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a80047d0 0x7fb2bc152d20 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fb2ac00e4b0 tx=0x7fb2ac00e970 comp rx=0 tx=0).stop 2023-10-16T08:40:49.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.833+0000 7fb2c1d58640 1 -- 172.21.15.40:0/637005239 shutdown_connections 2023-10-16T08:40:49.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.833+0000 7fb2c1d58640 1 --2- 172.21.15.40:0/637005239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb2bc0cad60 0x7fb2bc0cb210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:49.843 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.833+0000 7fb2c1d58640 1 --2- 172.21.15.40:0/637005239 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fb2bc14f0f0 0x7fb2bc0c9d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:49.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.833+0000 7fb2c1d58640 1 --2- 172.21.15.40:0/637005239 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fb29c0664d0 0x7fb29c068990 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:49.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.833+0000 7fb2c1d58640 1 --2- 172.21.15.40:0/637005239 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fb2a80047d0 0x7fb2bc152d20 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:49.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.833+0000 7fb2c1d58640 1 -- 172.21.15.40:0/637005239 >> 172.21.15.40:0/637005239 conn(0x7fb2bc06a290 msgr2=0x7fb2bc1599d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:49.844 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.833+0000 7fb2c1d58640 1 -- 172.21.15.40:0/637005239 shutdown_connections 2023-10-16T08:40:49.845 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:49.833+0000 7fb2c1d58640 1 -- 172.21.15.40:0/637005239 wait complete. 2023-10-16T08:40:49.856 INFO:teuthology.orchestra.run.smithi040.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2023-10-16T08:40:48.981745+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596433,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":220,"num_write_kb":2444,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":191,"ondisk_log_size":191,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219152,"kb_used_data":3024,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755568,"statfs":{"total":773094113280,"available":772869701632,"internally_reserved":0,"allocated":3096576,"data_stored":1859362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12743,"internal_metadata":221236793},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004075"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:14.566064+0000","last_change":"2023-10-16T08:40:14.566175+0000","last_active":"2023-10-16T08:40:14.566064+0000","last_peered":"2023-10-16T08:40:14.566064+0000","last_clean":"2023-10-16T08:40:14.566064+0000","last_became_active":"2023-10-16T08:39:26.962446+0000","last_became_peered":"2023-10-16T08:39:26.962446+0000","last_unstale":"2023-10-16T08:40:14.566064+0000","last_undegraded":"2023-10-16T08:40:14.566064+0000","last_fullsized":"2023-10-16T08:40:14.566064+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:14.565850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:14.565850+0000","last_clean_scrub_stamp":"2023-10-16T08:40:14.565850+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:48.417251+0000","scrub_duration":0.0038884470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:32.781386+0000","last_change":"2023-10-16T08:40:32.781487+0000","last_active":"2023-10-16T08:40:32.781386+0000","last_peered":"2023-10-16T08:40:32.781386+0000","last_clean":"2023-10-16T08:40:32.781386+0000","last_became_active":"2023-10-16T08:38:45.656884+0000","last_became_peered":"2023-10-16T08:38:45.656884+0000","last_unstale":"2023-10-16T08:40:32.781386+0000","last_undegraded":"2023-10-16T08:40:32.781386+0000","last_fullsized":"2023-10-16T08:40:32.781386+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:32.781167+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:32.781167+0000","last_clean_scrub_stamp":"2023-10-16T08:40:32.781167+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:43.619217+0000","scrub_duration":0.0037930300000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:06.160364+0000","last_change":"2023-10-16T08:40:06.160545+0000","last_active":"2023-10-16T08:40:06.160364+0000","last_peered":"2023-10-16T08:40:06.160364+0000","last_clean":"2023-10-16T08:40:06.160364+0000","last_became_active":"2023-10-16T08:38:45.629297+0000","last_became_peered":"2023-10-16T08:38:45.629297+0000","last_unstale":"2023-10-16T08:40:06.160364+0000","last_undegraded":"2023-10-16T08:40:06.160364+0000","last_fullsized":"2023-10-16T08:40:06.160364+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:06.160109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:06.160109+0000","last_clean_scrub_stamp":"2023-10-16T08:40:06.160109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:29.917626+0000","scrub_duration":0.003797534,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:10.594472+0000","last_change":"2023-10-16T08:40:10.594472+0000","last_active":"2023-10-16T08:40:10.594472+0000","last_peered":"2023-10-16T08:40:10.594472+0000","last_clean":"2023-10-16T08:40:10.594472+0000","last_became_active":"2023-10-16T08:38:45.675327+0000","last_became_peered":"2023-10-16T08:38:45.675327+0000","last_unstale":"2023-10-16T08:40:10.594472+0000","last_undegraded":"2023-10-16T08:40:10.594472+0000","last_fullsized":"2023-10-16T08:40:10.594472+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:10.594149+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:10.594149+0000","last_clean_scrub_stamp":"2023-10-16T08:40:10.594149+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:06:49.151485+0000","scrub_duration":0.003827324,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:10.247793+0000","last_change":"2023-10-16T08:40:10.247892+0000","last_active":"2023-10-16T08:40:10.247793+0000","last_peered":"2023-10-16T08:40:10.247793+0000","last_clean":"2023-10-16T08:40:10.247793+0000","last_became_active":"2023-10-16T08:38:45.652808+0000","last_became_peered":"2023-10-16T08:38:45.652808+0000","last_unstale":"2023-10-16T08:40:10.247793+0000","last_undegraded":"2023-10-16T08:40:10.247793+0000","last_fullsized":"2023-10-16T08:40:10.247793+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:10.247511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:10.247511+0000","last_clean_scrub_stamp":"2023-10-16T08:40:10.247511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:29:35.626689+0000","scrub_duration":0.0039935719999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:17.611393+0000","last_change":"2023-10-16T08:40:17.611491+0000","last_active":"2023-10-16T08:40:17.611393+0000","last_peered":"2023-10-16T08:40:17.611393+0000","last_clean":"2023-10-16T08:40:17.611393+0000","last_became_active":"2023-10-16T08:38:45.671146+0000","last_became_peered":"2023-10-16T08:38:45.671146+0000","last_unstale":"2023-10-16T08:40:17.611393+0000","last_undegraded":"2023-10-16T08:40:17.611393+0000","last_fullsized":"2023-10-16T08:40:17.611393+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:17.611169+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:17.611169+0000","last_clean_scrub_stamp":"2023-10-16T08:40:17.611169+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:52:37.226826+0000","scrub_duration":0.0038744809999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:25.379392+0000","last_change":"2023-10-16T08:40:25.379550+0000","last_active":"2023-10-16T08:40:25.379392+0000","last_peered":"2023-10-16T08:40:25.379392+0000","last_clean":"2023-10-16T08:40:25.379392+0000","last_became_active":"2023-10-16T08:38:45.651848+0000","last_became_peered":"2023-10-16T08:38:45.651848+0000","last_unstale":"2023-10-16T08:40:25.379392+0000","last_undegraded":"2023-10-16T08:40:25.379392+0000","last_fullsized":"2023-10-16T08:40:25.379392+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:25.379078+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:25.379078+0000","last_clean_scrub_stamp":"2023-10-16T08:40:25.379078+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:46:42.710112+0000","scrub_duration":0.0039533019999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:06.819154+0000","last_change":"2023-10-16T08:40:06.819332+0000","last_active":"2023-10-16T08:40:06.819154+0000","last_peered":"2023-10-16T08:40:06.819154+0000","last_clean":"2023-10-16T08:40:06.819154+0000","last_became_active":"2023-10-16T08:38:45.664702+0000","last_became_peered":"2023-10-16T08:38:45.664702+0000","last_unstale":"2023-10-16T08:40:06.819154+0000","last_undegraded":"2023-10-16T08:40:06.819154+0000","last_fullsized":"2023-10-16T08:40:06.819154+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:06.818902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:06.818902+0000","last_clean_scrub_stamp":"2023-10-16T08:40:06.818902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:00.868499+0000","scrub_duration":0.0035812309999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:13.209914+0000","last_change":"2023-10-16T08:40:13.210016+0000","last_active":"2023-10-16T08:40:13.209914+0000","last_peered":"2023-10-16T08:40:13.209914+0000","last_clean":"2023-10-16T08:40:13.209914+0000","last_became_active":"2023-10-16T08:38:45.666474+0000","last_became_peered":"2023-10-16T08:38:45.666474+0000","last_unstale":"2023-10-16T08:40:13.209914+0000","last_undegraded":"2023-10-16T08:40:13.209914+0000","last_fullsized":"2023-10-16T08:40:13.209914+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:13.209675+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:13.209675+0000","last_clean_scrub_stamp":"2023-10-16T08:40:13.209675+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:06:51.767012+0000","scrub_duration":0.0033902530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:15.995400+0000","last_change":"2023-10-16T08:40:15.995499+0000","last_active":"2023-10-16T08:40:15.995400+0000","last_peered":"2023-10-16T08:40:15.995400+0000","last_clean":"2023-10-16T08:40:15.995400+0000","last_became_active":"2023-10-16T08:38:45.668278+0000","last_became_peered":"2023-10-16T08:38:45.668278+0000","last_unstale":"2023-10-16T08:40:15.995400+0000","last_undegraded":"2023-10-16T08:40:15.995400+0000","last_fullsized":"2023-10-16T08:40:15.995400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:15.995163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:15.995163+0000","last_clean_scrub_stamp":"2023-10-16T08:40:15.995163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:04:59.725405+0000","scrub_duration":0.0041064329999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:12.005283+0000","last_change":"2023-10-16T08:40:12.005399+0000","last_active":"2023-10-16T08:40:12.005283+0000","last_peered":"2023-10-16T08:40:12.005283+0000","last_clean":"2023-10-16T08:40:12.005283+0000","last_became_active":"2023-10-16T08:38:45.663813+0000","last_became_peered":"2023-10-16T08:38:45.663813+0000","last_unstale":"2023-10-16T08:40:12.005283+0000","last_undegraded":"2023-10-16T08:40:12.005283+0000","last_fullsized":"2023-10-16T08:40:12.005283+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:12.005041+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:12.005041+0000","last_clean_scrub_stamp":"2023-10-16T08:40:12.005041+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:12:17.822987+0000","scrub_duration":0.0037935099999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:58.863583+0000","last_change":"2023-10-16T08:39:58.863718+0000","last_active":"2023-10-16T08:39:58.863583+0000","last_peered":"2023-10-16T08:39:58.863583+0000","last_clean":"2023-10-16T08:39:58.863583+0000","last_became_active":"2023-10-16T08:38:45.675320+0000","last_became_peered":"2023-10-16T08:38:45.675320+0000","last_unstale":"2023-10-16T08:39:58.863583+0000","last_undegraded":"2023-10-16T08:39:58.863583+0000","last_fullsized":"2023-10-16T08:39:58.863583+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:58.863355+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:58.863355+0000","last_clean_scrub_stamp":"2023-10-16T08:39:58.863355+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:17:12.041514+0000","scrub_duration":0.0033670039999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:19.634096+0000","last_change":"2023-10-16T08:40:19.634287+0000","last_active":"2023-10-16T08:40:19.634096+0000","last_peered":"2023-10-16T08:40:19.634096+0000","last_clean":"2023-10-16T08:40:19.634096+0000","last_became_active":"2023-10-16T08:38:45.667428+0000","last_became_peered":"2023-10-16T08:38:45.667428+0000","last_unstale":"2023-10-16T08:40:19.634096+0000","last_undegraded":"2023-10-16T08:40:19.634096+0000","last_fullsized":"2023-10-16T08:40:19.634096+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:19.633788+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:19.633788+0000","last_clean_scrub_stamp":"2023-10-16T08:40:19.633788+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:10:51.712948+0000","scrub_duration":0.0043199290000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:50.221350+0000","last_change":"2023-10-16T08:39:50.221441+0000","last_active":"2023-10-16T08:39:50.221350+0000","last_peered":"2023-10-16T08:39:50.221350+0000","last_clean":"2023-10-16T08:39:50.221350+0000","last_became_active":"2023-10-16T08:38:45.636785+0000","last_became_peered":"2023-10-16T08:38:45.636785+0000","last_unstale":"2023-10-16T08:39:50.221350+0000","last_undegraded":"2023-10-16T08:39:50.221350+0000","last_fullsized":"2023-10-16T08:39:50.221350+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:50.221128+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:50.221128+0000","last_clean_scrub_stamp":"2023-10-16T08:39:50.221128+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:58:43.521146+0000","scrub_duration":0.002811554,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:20.751761+0000","last_change":"2023-10-16T08:40:20.751864+0000","last_active":"2023-10-16T08:40:20.751761+0000","last_peered":"2023-10-16T08:40:20.751761+0000","last_clean":"2023-10-16T08:40:20.751761+0000","last_became_active":"2023-10-16T08:38:45.655395+0000","last_became_peered":"2023-10-16T08:38:45.655395+0000","last_unstale":"2023-10-16T08:40:20.751761+0000","last_undegraded":"2023-10-16T08:40:20.751761+0000","last_fullsized":"2023-10-16T08:40:20.751761+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:20.751504+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:20.751504+0000","last_clean_scrub_stamp":"2023-10-16T08:40:20.751504+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:14.801101+0000","scrub_duration":0.0037562749999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:04.505262+0000","last_change":"2023-10-16T08:40:04.505370+0000","last_active":"2023-10-16T08:40:04.505262+0000","last_peered":"2023-10-16T08:40:04.505262+0000","last_clean":"2023-10-16T08:40:04.505262+0000","last_became_active":"2023-10-16T08:38:45.663312+0000","last_became_peered":"2023-10-16T08:38:45.663312+0000","last_unstale":"2023-10-16T08:40:04.505262+0000","last_undegraded":"2023-10-16T08:40:04.505262+0000","last_fullsized":"2023-10-16T08:40:04.505262+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:04.505050+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:04.505050+0000","last_clean_scrub_stamp":"2023-10-16T08:40:04.505050+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:15.343100+0000","scrub_duration":0.0035649169999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:13.571796+0000","last_change":"2023-10-16T08:40:13.571890+0000","last_active":"2023-10-16T08:40:13.571796+0000","last_peered":"2023-10-16T08:40:13.571796+0000","last_clean":"2023-10-16T08:40:13.571796+0000","last_became_active":"2023-10-16T08:39:27.012068+0000","last_became_peered":"2023-10-16T08:39:27.012068+0000","last_unstale":"2023-10-16T08:40:13.571796+0000","last_undegraded":"2023-10-16T08:40:13.571796+0000","last_fullsized":"2023-10-16T08:40:13.571796+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:13.571587+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:13.571587+0000","last_clean_scrub_stamp":"2023-10-16T08:40:13.571587+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:55:53.091091+0000","scrub_duration":0.0036468709999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:05.650748+0000","last_change":"2023-10-16T08:40:05.650848+0000","last_active":"2023-10-16T08:40:05.650748+0000","last_peered":"2023-10-16T08:40:05.650748+0000","last_clean":"2023-10-16T08:40:05.650748+0000","last_became_active":"2023-10-16T08:38:45.630641+0000","last_became_peered":"2023-10-16T08:38:45.630641+0000","last_unstale":"2023-10-16T08:40:05.650748+0000","last_undegraded":"2023-10-16T08:40:05.650748+0000","last_fullsized":"2023-10-16T08:40:05.650748+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:05.650492+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:05.650492+0000","last_clean_scrub_stamp":"2023-10-16T08:40:05.650492+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:42:23.950253+0000","scrub_duration":0.0038292840000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:43.348965+0000","last_change":"2023-10-16T08:40:43.349080+0000","last_active":"2023-10-16T08:40:43.348965+0000","last_peered":"2023-10-16T08:40:43.348965+0000","last_clean":"2023-10-16T08:40:43.348965+0000","last_became_active":"2023-10-16T08:38:45.649475+0000","last_became_peered":"2023-10-16T08:38:45.649475+0000","last_unstale":"2023-10-16T08:40:43.348965+0000","last_undegraded":"2023-10-16T08:40:43.348965+0000","last_fullsized":"2023-10-16T08:40:43.348965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:43.348744+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:43.348744+0000","last_clean_scrub_stamp":"2023-10-16T08:40:43.348744+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:49:58.051421+0000","scrub_duration":0.0041793469999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:18.946495+0000","last_change":"2023-10-16T08:40:18.946701+0000","last_active":"2023-10-16T08:40:18.946495+0000","last_peered":"2023-10-16T08:40:18.946495+0000","last_clean":"2023-10-16T08:40:18.946495+0000","last_became_active":"2023-10-16T08:38:45.665291+0000","last_became_peered":"2023-10-16T08:38:45.665291+0000","last_unstale":"2023-10-16T08:40:18.946495+0000","last_undegraded":"2023-10-16T08:40:18.946495+0000","last_fullsized":"2023-10-16T08:40:18.946495+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:18.946250+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:18.946250+0000","last_clean_scrub_stamp":"2023-10-16T08:40:18.946250+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:46:43.621078+0000","scrub_duration":0.00352601,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:04.141444+0000","last_change":"2023-10-16T08:40:04.141546+0000","last_active":"2023-10-16T08:40:04.141444+0000","last_peered":"2023-10-16T08:40:04.141444+0000","last_clean":"2023-10-16T08:40:04.141444+0000","last_became_active":"2023-10-16T08:38:45.652412+0000","last_became_peered":"2023-10-16T08:38:45.652412+0000","last_unstale":"2023-10-16T08:40:04.141444+0000","last_undegraded":"2023-10-16T08:40:04.141444+0000","last_fullsized":"2023-10-16T08:40:04.141444+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:04.141224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:04.141224+0000","last_clean_scrub_stamp":"2023-10-16T08:40:04.141224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:07.519281+0000","scrub_duration":0.0035640720000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:05.502168+0000","last_change":"2023-10-16T08:40:05.502348+0000","last_active":"2023-10-16T08:40:05.502168+0000","last_peered":"2023-10-16T08:40:05.502168+0000","last_clean":"2023-10-16T08:40:05.502168+0000","last_became_active":"2023-10-16T08:38:45.676844+0000","last_became_peered":"2023-10-16T08:38:45.676844+0000","last_unstale":"2023-10-16T08:40:05.502168+0000","last_undegraded":"2023-10-16T08:40:05.502168+0000","last_fullsized":"2023-10-16T08:40:05.502168+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:05.501915+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:05.501915+0000","last_clean_scrub_stamp":"2023-10-16T08:40:05.501915+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:46:30.176743+0000","scrub_duration":0.0034663509999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:05.864255+0000","last_change":"2023-10-16T08:40:05.864353+0000","last_active":"2023-10-16T08:40:05.864255+0000","last_peered":"2023-10-16T08:40:05.864255+0000","last_clean":"2023-10-16T08:40:05.864255+0000","last_became_active":"2023-10-16T08:38:45.662944+0000","last_became_peered":"2023-10-16T08:38:45.662944+0000","last_unstale":"2023-10-16T08:40:05.864255+0000","last_undegraded":"2023-10-16T08:40:05.864255+0000","last_fullsized":"2023-10-16T08:40:05.864255+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:05.864043+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:05.864043+0000","last_clean_scrub_stamp":"2023-10-16T08:40:05.864043+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:39.715444+0000","scrub_duration":0.0038577170000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:00.546243+0000","last_change":"2023-10-16T08:40:00.546291+0000","last_active":"2023-10-16T08:40:00.546243+0000","last_peered":"2023-10-16T08:40:00.546243+0000","last_clean":"2023-10-16T08:40:00.546243+0000","last_became_active":"2023-10-16T08:38:45.666641+0000","last_became_peered":"2023-10-16T08:38:45.666641+0000","last_unstale":"2023-10-16T08:40:00.546243+0000","last_undegraded":"2023-10-16T08:40:00.546243+0000","last_fullsized":"2023-10-16T08:40:00.546243+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:00.546142+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:00.546142+0000","last_clean_scrub_stamp":"2023-10-16T08:40:00.546142+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:13:33.731843+0000","scrub_duration":0.0032222819999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:28.562635+0000","last_change":"2023-10-16T08:40:28.562752+0000","last_active":"2023-10-16T08:40:28.562635+0000","last_peered":"2023-10-16T08:40:28.562635+0000","last_clean":"2023-10-16T08:40:28.562635+0000","last_became_active":"2023-10-16T08:38:45.669734+0000","last_became_peered":"2023-10-16T08:38:45.669734+0000","last_unstale":"2023-10-16T08:40:28.562635+0000","last_undegraded":"2023-10-16T08:40:28.562635+0000","last_fullsized":"2023-10-16T08:40:28.562635+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:28.562406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:28.562406+0000","last_clean_scrub_stamp":"2023-10-16T08:40:28.562406+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:24:33.540852+0000","scrub_duration":0.0038215530000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:07.003998+0000","last_change":"2023-10-16T08:40:07.004096+0000","last_active":"2023-10-16T08:40:07.003998+0000","last_peered":"2023-10-16T08:40:07.003998+0000","last_clean":"2023-10-16T08:40:07.003998+0000","last_became_active":"2023-10-16T08:38:45.665972+0000","last_became_peered":"2023-10-16T08:38:45.665972+0000","last_unstale":"2023-10-16T08:40:07.003998+0000","last_undegraded":"2023-10-16T08:40:07.003998+0000","last_fullsized":"2023-10-16T08:40:07.003998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:07.003796+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:07.003796+0000","last_clean_scrub_stamp":"2023-10-16T08:40:07.003796+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:01.053394+0000","scrub_duration":0.0034323180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:36.784674+0000","last_change":"2023-10-16T08:40:36.784784+0000","last_active":"2023-10-16T08:40:36.784674+0000","last_peered":"2023-10-16T08:40:36.784674+0000","last_clean":"2023-10-16T08:40:36.784674+0000","last_became_active":"2023-10-16T08:38:45.666403+0000","last_became_peered":"2023-10-16T08:38:45.666403+0000","last_unstale":"2023-10-16T08:40:36.784674+0000","last_undegraded":"2023-10-16T08:40:36.784674+0000","last_fullsized":"2023-10-16T08:40:36.784674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:36.784386+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:36.784386+0000","last_clean_scrub_stamp":"2023-10-16T08:40:36.784386+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:47:01.459214+0000","scrub_duration":0.0036700359999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:04.840228+0000","last_change":"2023-10-16T08:40:04.840318+0000","last_active":"2023-10-16T08:40:04.840228+0000","last_peered":"2023-10-16T08:40:04.840228+0000","last_clean":"2023-10-16T08:40:04.840228+0000","last_became_active":"2023-10-16T08:38:45.664572+0000","last_became_peered":"2023-10-16T08:38:45.664572+0000","last_unstale":"2023-10-16T08:40:04.840228+0000","last_undegraded":"2023-10-16T08:40:04.840228+0000","last_fullsized":"2023-10-16T08:40:04.840228+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:04.840012+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:04.840012+0000","last_clean_scrub_stamp":"2023-10-16T08:40:04.840012+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:55:44.359515+0000","scrub_duration":0.0038448359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:14.056731+0000","last_change":"2023-10-16T08:40:14.056845+0000","last_active":"2023-10-16T08:40:14.056731+0000","last_peered":"2023-10-16T08:40:14.056731+0000","last_clean":"2023-10-16T08:40:14.056731+0000","last_became_active":"2023-10-16T08:38:45.666547+0000","last_became_peered":"2023-10-16T08:38:45.666547+0000","last_unstale":"2023-10-16T08:40:14.056731+0000","last_undegraded":"2023-10-16T08:40:14.056731+0000","last_fullsized":"2023-10-16T08:40:14.056731+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:14.056503+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:14.056503+0000","last_clean_scrub_stamp":"2023-10-16T08:40:14.056503+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T18:36:08.106100+0000","scrub_duration":0.0033984269999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":19,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:11.524811+0000","last_change":"2023-10-16T08:40:11.524934+0000","last_active":"2023-10-16T08:40:11.524811+0000","last_peered":"2023-10-16T08:40:11.524811+0000","last_clean":"2023-10-16T08:40:11.524811+0000","last_became_active":"2023-10-16T08:39:27.009983+0000","last_became_peered":"2023-10-16T08:39:27.009983+0000","last_unstale":"2023-10-16T08:40:11.524811+0000","last_undegraded":"2023-10-16T08:40:11.524811+0000","last_fullsized":"2023-10-16T08:40:11.524811+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:11.524600+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:11.524600+0000","last_clean_scrub_stamp":"2023-10-16T08:40:11.524600+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:50:05.973297+0000","scrub_duration":0.003632943,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":29,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:58.646552+0000","last_change":"2023-10-16T08:39:58.646655+0000","last_active":"2023-10-16T08:39:58.646552+0000","last_peered":"2023-10-16T08:39:58.646552+0000","last_clean":"2023-10-16T08:39:58.646552+0000","last_became_active":"2023-10-16T08:39:27.013658+0000","last_became_peered":"2023-10-16T08:39:27.013658+0000","last_unstale":"2023-10-16T08:39:58.646552+0000","last_undegraded":"2023-10-16T08:39:58.646552+0000","last_fullsized":"2023-10-16T08:39:58.646552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:58.646339+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:58.646339+0000","last_clean_scrub_stamp":"2023-10-16T08:39:58.646339+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:32:08.951921+0000","scrub_duration":0.0036955680000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:02.119406+0000","last_change":"2023-10-16T08:40:02.119578+0000","last_active":"2023-10-16T08:40:02.119406+0000","last_peered":"2023-10-16T08:40:02.119406+0000","last_clean":"2023-10-16T08:40:02.119406+0000","last_became_active":"2023-10-16T08:38:45.636762+0000","last_became_peered":"2023-10-16T08:38:45.636762+0000","last_unstale":"2023-10-16T08:40:02.119406+0000","last_undegraded":"2023-10-16T08:40:02.119406+0000","last_fullsized":"2023-10-16T08:40:02.119406+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:02.119151+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:02.119151+0000","last_clean_scrub_stamp":"2023-10-16T08:40:02.119151+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:23:40.725505+0000","scrub_duration":0.00382139,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:52.785339+0000","last_change":"2023-10-16T08:39:52.785439+0000","last_active":"2023-10-16T08:39:52.785339+0000","last_peered":"2023-10-16T08:39:52.785339+0000","last_clean":"2023-10-16T08:39:52.785339+0000","last_became_active":"2023-10-16T08:38:45.658808+0000","last_became_peered":"2023-10-16T08:38:45.658808+0000","last_unstale":"2023-10-16T08:39:52.785339+0000","last_undegraded":"2023-10-16T08:39:52.785339+0000","last_fullsized":"2023-10-16T08:39:52.785339+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:52.785111+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:52.785111+0000","last_clean_scrub_stamp":"2023-10-16T08:39:52.785111+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:55:48.998080+0000","scrub_duration":0.0036557069999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"19'1","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:48.153116+0000","last_change":"2023-10-16T08:39:48.153181+0000","last_active":"2023-10-16T08:39:48.153116+0000","last_peered":"2023-10-16T08:39:48.153116+0000","last_clean":"2023-10-16T08:39:48.153116+0000","last_became_active":"2023-10-16T08:38:44.643614+0000","last_became_peered":"2023-10-16T08:38:44.643614+0000","last_unstale":"2023-10-16T08:39:48.153116+0000","last_undegraded":"2023-10-16T08:39:48.153116+0000","last_fullsized":"2023-10-16T08:39:48.153116+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"19'1","last_scrub_stamp":"2023-10-16T08:39:48.153004+0000","last_deep_scrub":"19'1","last_deep_scrub_stamp":"2023-10-16T08:39:48.153004+0000","last_clean_scrub_stamp":"2023-10-16T08:39:48.153004+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:20.737033+0000","scrub_duration":0.002884073,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:26.791436+0000","last_change":"2023-10-16T08:40:26.791622+0000","last_active":"2023-10-16T08:40:26.791436+0000","last_peered":"2023-10-16T08:40:26.791436+0000","last_clean":"2023-10-16T08:40:26.791436+0000","last_became_active":"2023-10-16T08:38:45.658409+0000","last_became_peered":"2023-10-16T08:38:45.658409+0000","last_unstale":"2023-10-16T08:40:26.791436+0000","last_undegraded":"2023-10-16T08:40:26.791436+0000","last_fullsized":"2023-10-16T08:40:26.791436+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:26.791165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:26.791165+0000","last_clean_scrub_stamp":"2023-10-16T08:40:26.791165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:37.629215+0000","scrub_duration":0.0038734239999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"19'1","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:00.189167+0000","last_change":"2023-10-16T08:40:00.189264+0000","last_active":"2023-10-16T08:40:00.189167+0000","last_peered":"2023-10-16T08:40:00.189167+0000","last_clean":"2023-10-16T08:40:00.189167+0000","last_became_active":"2023-10-16T08:38:44.658257+0000","last_became_peered":"2023-10-16T08:38:44.658257+0000","last_unstale":"2023-10-16T08:40:00.189167+0000","last_undegraded":"2023-10-16T08:40:00.189167+0000","last_fullsized":"2023-10-16T08:40:00.189167+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"19'1","last_scrub_stamp":"2023-10-16T08:40:00.188936+0000","last_deep_scrub":"19'1","last_deep_scrub_stamp":"2023-10-16T08:40:00.188936+0000","last_clean_scrub_stamp":"2023-10-16T08:40:00.188936+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T13:17:03.566993+0000","scrub_duration":0.0036410499999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:51.092373+0000","last_change":"2023-10-16T08:39:51.092469+0000","last_active":"2023-10-16T08:39:51.092373+0000","last_peered":"2023-10-16T08:39:51.092373+0000","last_clean":"2023-10-16T08:39:51.092373+0000","last_became_active":"2023-10-16T08:38:44.645931+0000","last_became_peered":"2023-10-16T08:38:44.645931+0000","last_unstale":"2023-10-16T08:39:51.092373+0000","last_undegraded":"2023-10-16T08:39:51.092373+0000","last_fullsized":"2023-10-16T08:39:51.092373+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:51.092105+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:51.092105+0000","last_clean_scrub_stamp":"2023-10-16T08:39:51.092105+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:32:01.397687+0000","scrub_duration":0.0039256869999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:01.915584+0000","last_change":"2023-10-16T08:40:01.915678+0000","last_active":"2023-10-16T08:40:01.915584+0000","last_peered":"2023-10-16T08:40:01.915584+0000","last_clean":"2023-10-16T08:40:01.915584+0000","last_became_active":"2023-10-16T08:38:45.662072+0000","last_became_peered":"2023-10-16T08:38:45.662072+0000","last_unstale":"2023-10-16T08:40:01.915584+0000","last_undegraded":"2023-10-16T08:40:01.915584+0000","last_fullsized":"2023-10-16T08:40:01.915584+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:01.915369+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:01.915369+0000","last_clean_scrub_stamp":"2023-10-16T08:40:01.915369+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:49:56.364066+0000","scrub_duration":0.0034239230000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:50.531962+0000","last_change":"2023-10-16T08:39:50.532059+0000","last_active":"2023-10-16T08:39:50.531962+0000","last_peered":"2023-10-16T08:39:50.531962+0000","last_clean":"2023-10-16T08:39:50.531962+0000","last_became_active":"2023-10-16T08:38:44.657065+0000","last_became_peered":"2023-10-16T08:38:44.657065+0000","last_unstale":"2023-10-16T08:39:50.531962+0000","last_undegraded":"2023-10-16T08:39:50.531962+0000","last_fullsized":"2023-10-16T08:39:50.531962+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:50.531739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:50.531739+0000","last_clean_scrub_stamp":"2023-10-16T08:39:50.531739+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:49:44.980436+0000","scrub_duration":0.0034347560000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:40.692201+0000","last_change":"2023-10-16T08:40:40.692299+0000","last_active":"2023-10-16T08:40:40.692201+0000","last_peered":"2023-10-16T08:40:40.692201+0000","last_clean":"2023-10-16T08:40:40.692201+0000","last_became_active":"2023-10-16T08:38:45.667413+0000","last_became_peered":"2023-10-16T08:38:45.667413+0000","last_unstale":"2023-10-16T08:40:40.692201+0000","last_undegraded":"2023-10-16T08:40:40.692201+0000","last_fullsized":"2023-10-16T08:40:40.692201+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:40.691955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:40.691955+0000","last_clean_scrub_stamp":"2023-10-16T08:40:40.691955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:30:40.773315+0000","scrub_duration":0.0036545470000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:53.126900+0000","last_change":"2023-10-16T08:39:53.127012+0000","last_active":"2023-10-16T08:39:53.126900+0000","last_peered":"2023-10-16T08:39:53.126900+0000","last_clean":"2023-10-16T08:39:53.126900+0000","last_became_active":"2023-10-16T08:38:44.662383+0000","last_became_peered":"2023-10-16T08:38:44.662383+0000","last_unstale":"2023-10-16T08:39:53.126900+0000","last_undegraded":"2023-10-16T08:39:53.126900+0000","last_fullsized":"2023-10-16T08:39:53.126900+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:53.126641+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:53.126641+0000","last_clean_scrub_stamp":"2023-10-16T08:39:53.126641+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:32:03.432222+0000","scrub_duration":0.0040293320000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:15.602153+0000","last_change":"2023-10-16T08:40:15.602249+0000","last_active":"2023-10-16T08:40:15.602153+0000","last_peered":"2023-10-16T08:40:15.602153+0000","last_clean":"2023-10-16T08:40:15.602153+0000","last_became_active":"2023-10-16T08:38:45.678050+0000","last_became_peered":"2023-10-16T08:38:45.678050+0000","last_unstale":"2023-10-16T08:40:15.602153+0000","last_undegraded":"2023-10-16T08:40:15.602153+0000","last_fullsized":"2023-10-16T08:40:15.602153+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:15.601899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:15.601899+0000","last_clean_scrub_stamp":"2023-10-16T08:40:15.601899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:19:06.907714+0000","scrub_duration":0.003337497,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:51.509756+0000","last_change":"2023-10-16T08:39:51.509855+0000","last_active":"2023-10-16T08:39:51.509756+0000","last_peered":"2023-10-16T08:39:51.509756+0000","last_clean":"2023-10-16T08:39:51.509756+0000","last_became_active":"2023-10-16T08:38:44.652618+0000","last_became_peered":"2023-10-16T08:38:44.652618+0000","last_unstale":"2023-10-16T08:39:51.509756+0000","last_undegraded":"2023-10-16T08:39:51.509756+0000","last_fullsized":"2023-10-16T08:39:51.509756+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:51.509535+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:51.509535+0000","last_clean_scrub_stamp":"2023-10-16T08:39:51.509535+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:55:31.029038+0000","scrub_duration":0.0032865199999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:28.837882+0000","last_change":"2023-10-16T08:40:28.837986+0000","last_active":"2023-10-16T08:40:28.837882+0000","last_peered":"2023-10-16T08:40:28.837882+0000","last_clean":"2023-10-16T08:40:28.837882+0000","last_became_active":"2023-10-16T08:38:45.655213+0000","last_became_peered":"2023-10-16T08:38:45.655213+0000","last_unstale":"2023-10-16T08:40:28.837882+0000","last_undegraded":"2023-10-16T08:40:28.837882+0000","last_fullsized":"2023-10-16T08:40:28.837882+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:28.837650+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:28.837650+0000","last_clean_scrub_stamp":"2023-10-16T08:40:28.837650+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:38:18.613428+0000","scrub_duration":0.0037546760000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"22'5","reported_seq":28,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:11.675792+0000","last_change":"2023-10-16T08:40:11.675907+0000","last_active":"2023-10-16T08:40:11.675792+0000","last_peered":"2023-10-16T08:40:11.675792+0000","last_clean":"2023-10-16T08:40:11.675792+0000","last_became_active":"2023-10-16T08:38:44.651359+0000","last_became_peered":"2023-10-16T08:38:44.651359+0000","last_unstale":"2023-10-16T08:40:11.675792+0000","last_undegraded":"2023-10-16T08:40:11.675792+0000","last_fullsized":"2023-10-16T08:40:11.675792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'5","last_scrub_stamp":"2023-10-16T08:40:11.675566+0000","last_deep_scrub":"22'5","last_deep_scrub_stamp":"2023-10-16T08:40:11.675566+0000","last_clean_scrub_stamp":"2023-10-16T08:40:11.675566+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:23:50.281919+0000","scrub_duration":0.0091403179999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:54.792955+0000","last_change":"2023-10-16T08:39:54.793071+0000","last_active":"2023-10-16T08:39:54.792955+0000","last_peered":"2023-10-16T08:39:54.792955+0000","last_clean":"2023-10-16T08:39:54.792955+0000","last_became_active":"2023-10-16T08:38:45.660611+0000","last_became_peered":"2023-10-16T08:38:45.660611+0000","last_unstale":"2023-10-16T08:39:54.792955+0000","last_undegraded":"2023-10-16T08:39:54.792955+0000","last_fullsized":"2023-10-16T08:39:54.792955+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:54.792721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:54.792721+0000","last_clean_scrub_stamp":"2023-10-16T08:39:54.792721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T16:03:27.376750+0000","scrub_duration":0.0038456089999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"22'20","reported_seq":43,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:59.159576+0000","last_change":"2023-10-16T08:39:59.159729+0000","last_active":"2023-10-16T08:39:59.159576+0000","last_peered":"2023-10-16T08:39:59.159576+0000","last_clean":"2023-10-16T08:39:59.159576+0000","last_became_active":"2023-10-16T08:38:44.647001+0000","last_became_peered":"2023-10-16T08:38:44.647001+0000","last_unstale":"2023-10-16T08:39:59.159576+0000","last_undegraded":"2023-10-16T08:39:59.159576+0000","last_fullsized":"2023-10-16T08:39:59.159576+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'20","last_scrub_stamp":"2023-10-16T08:39:59.159359+0000","last_deep_scrub":"22'20","last_deep_scrub_stamp":"2023-10-16T08:39:59.159359+0000","last_clean_scrub_stamp":"2023-10-16T08:39:59.159359+0000","objects_scrubbed":4,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:42:17.459120+0000","scrub_duration":0.0099116120000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5267,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:09.284560+0000","last_change":"2023-10-16T08:40:09.284656+0000","last_active":"2023-10-16T08:40:09.284560+0000","last_peered":"2023-10-16T08:40:09.284560+0000","last_clean":"2023-10-16T08:40:09.284560+0000","last_became_active":"2023-10-16T08:38:45.667334+0000","last_became_peered":"2023-10-16T08:38:45.667334+0000","last_unstale":"2023-10-16T08:40:09.284560+0000","last_undegraded":"2023-10-16T08:40:09.284560+0000","last_fullsized":"2023-10-16T08:40:09.284560+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:09.284344+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:09.284344+0000","last_clean_scrub_stamp":"2023-10-16T08:40:09.284344+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:37:59.060122+0000","scrub_duration":0.0038307739999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"22'4","reported_seq":27,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:46.198227+0000","last_change":"2023-10-16T08:39:46.198310+0000","last_active":"2023-10-16T08:39:46.198227+0000","last_peered":"2023-10-16T08:39:46.198227+0000","last_clean":"2023-10-16T08:39:46.198227+0000","last_became_active":"2023-10-16T08:38:44.655469+0000","last_became_peered":"2023-10-16T08:38:44.655469+0000","last_unstale":"2023-10-16T08:39:46.198227+0000","last_undegraded":"2023-10-16T08:39:46.198227+0000","last_fullsized":"2023-10-16T08:39:46.198227+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'4","last_scrub_stamp":"2023-10-16T08:39:46.198102+0000","last_deep_scrub":"22'4","last_deep_scrub_stamp":"2023-10-16T08:39:46.198102+0000","last_clean_scrub_stamp":"2023-10-16T08:39:46.198102+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:19:00.034900+0000","scrub_duration":0.00664833,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:12.071405+0000","last_change":"2023-10-16T08:40:12.071542+0000","last_active":"2023-10-16T08:40:12.071405+0000","last_peered":"2023-10-16T08:40:12.071405+0000","last_clean":"2023-10-16T08:40:12.071405+0000","last_became_active":"2023-10-16T08:38:45.650563+0000","last_became_peered":"2023-10-16T08:38:45.650563+0000","last_unstale":"2023-10-16T08:40:12.071405+0000","last_undegraded":"2023-10-16T08:40:12.071405+0000","last_fullsized":"2023-10-16T08:40:12.071405+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:12.071182+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:12.071182+0000","last_clean_scrub_stamp":"2023-10-16T08:40:12.071182+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:24.423790+0000","scrub_duration":0.0042431159999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:56.567995+0000","last_change":"2023-10-16T08:39:56.568099+0000","last_active":"2023-10-16T08:39:56.567995+0000","last_peered":"2023-10-16T08:39:56.567995+0000","last_clean":"2023-10-16T08:39:56.567995+0000","last_became_active":"2023-10-16T08:38:44.662110+0000","last_became_peered":"2023-10-16T08:38:44.662110+0000","last_unstale":"2023-10-16T08:39:56.567995+0000","last_undegraded":"2023-10-16T08:39:56.567995+0000","last_fullsized":"2023-10-16T08:39:56.567995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:56.567779+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:56.567779+0000","last_clean_scrub_stamp":"2023-10-16T08:39:56.567779+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:37:13.876495+0000","scrub_duration":0.0037362279999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:20.235447+0000","last_change":"2023-10-16T08:40:20.235539+0000","last_active":"2023-10-16T08:40:20.235447+0000","last_peered":"2023-10-16T08:40:20.235447+0000","last_clean":"2023-10-16T08:40:20.235447+0000","last_became_active":"2023-10-16T08:38:45.661595+0000","last_became_peered":"2023-10-16T08:38:45.661595+0000","last_unstale":"2023-10-16T08:40:20.235447+0000","last_undegraded":"2023-10-16T08:40:20.235447+0000","last_fullsized":"2023-10-16T08:40:20.235447+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:20.235213+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:20.235213+0000","last_clean_scrub_stamp":"2023-10-16T08:40:20.235213+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:37:01.456143+0000","scrub_duration":0.0035930620000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:50.191434+0000","last_change":"2023-10-16T08:39:50.191539+0000","last_active":"2023-10-16T08:39:50.191434+0000","last_peered":"2023-10-16T08:39:50.191434+0000","last_clean":"2023-10-16T08:39:50.191434+0000","last_became_active":"2023-10-16T08:38:44.657153+0000","last_became_peered":"2023-10-16T08:38:44.657153+0000","last_unstale":"2023-10-16T08:39:50.191434+0000","last_undegraded":"2023-10-16T08:39:50.191434+0000","last_fullsized":"2023-10-16T08:39:50.191434+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:50.191165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:50.191165+0000","last_clean_scrub_stamp":"2023-10-16T08:39:50.191165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:31:15.292095+0000","scrub_duration":0.0038904819999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:04.049549+0000","last_change":"2023-10-16T08:40:04.049648+0000","last_active":"2023-10-16T08:40:04.049549+0000","last_peered":"2023-10-16T08:40:04.049549+0000","last_clean":"2023-10-16T08:40:04.049549+0000","last_became_active":"2023-10-16T08:38:45.635633+0000","last_became_peered":"2023-10-16T08:38:45.635633+0000","last_unstale":"2023-10-16T08:40:04.049549+0000","last_undegraded":"2023-10-16T08:40:04.049549+0000","last_fullsized":"2023-10-16T08:40:04.049549+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:04.049337+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:04.049337+0000","last_clean_scrub_stamp":"2023-10-16T08:40:04.049337+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:02:16.401945+0000","scrub_duration":0.0038362539999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"22'12","reported_seq":35,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:46.606020+0000","last_change":"2023-10-16T08:39:46.606097+0000","last_active":"2023-10-16T08:39:46.606020+0000","last_peered":"2023-10-16T08:39:46.606020+0000","last_clean":"2023-10-16T08:39:46.606020+0000","last_became_active":"2023-10-16T08:38:44.637640+0000","last_became_peered":"2023-10-16T08:38:44.637640+0000","last_unstale":"2023-10-16T08:39:46.606020+0000","last_undegraded":"2023-10-16T08:39:46.606020+0000","last_fullsized":"2023-10-16T08:39:46.606020+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'12","last_scrub_stamp":"2023-10-16T08:39:46.605891+0000","last_deep_scrub":"22'12","last_deep_scrub_stamp":"2023-10-16T08:39:46.605891+0000","last_clean_scrub_stamp":"2023-10-16T08:39:46.605891+0000","objects_scrubbed":4,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:02.027806+0000","scrub_duration":0.0068877740000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:21.278804+0000","last_change":"2023-10-16T08:40:21.278917+0000","last_active":"2023-10-16T08:40:21.278804+0000","last_peered":"2023-10-16T08:40:21.278804+0000","last_clean":"2023-10-16T08:40:21.278804+0000","last_became_active":"2023-10-16T08:38:45.656651+0000","last_became_peered":"2023-10-16T08:38:45.656651+0000","last_unstale":"2023-10-16T08:40:21.278804+0000","last_undegraded":"2023-10-16T08:40:21.278804+0000","last_fullsized":"2023-10-16T08:40:21.278804+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:21.278543+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:21.278543+0000","last_clean_scrub_stamp":"2023-10-16T08:40:21.278543+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:19:12.584357+0000","scrub_duration":0.0037387620000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"22'2","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:47.765260+0000","last_change":"2023-10-16T08:39:47.765365+0000","last_active":"2023-10-16T08:39:47.765260+0000","last_peered":"2023-10-16T08:39:47.765260+0000","last_clean":"2023-10-16T08:39:47.765260+0000","last_became_active":"2023-10-16T08:38:44.647887+0000","last_became_peered":"2023-10-16T08:38:44.647887+0000","last_unstale":"2023-10-16T08:39:47.765260+0000","last_undegraded":"2023-10-16T08:39:47.765260+0000","last_fullsized":"2023-10-16T08:39:47.765260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'2","last_scrub_stamp":"2023-10-16T08:39:47.765049+0000","last_deep_scrub":"22'2","last_deep_scrub_stamp":"2023-10-16T08:39:47.765049+0000","last_clean_scrub_stamp":"2023-10-16T08:39:47.765049+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:16:03.642054+0000","scrub_duration":0.0054068919999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:49.280907+0000","last_change":"2023-10-16T08:39:49.281008+0000","last_active":"2023-10-16T08:39:49.280907+0000","last_peered":"2023-10-16T08:39:49.280907+0000","last_clean":"2023-10-16T08:39:49.280907+0000","last_became_active":"2023-10-16T08:38:45.656750+0000","last_became_peered":"2023-10-16T08:38:45.656750+0000","last_unstale":"2023-10-16T08:39:49.280907+0000","last_undegraded":"2023-10-16T08:39:49.280907+0000","last_fullsized":"2023-10-16T08:39:49.280907+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:49.280644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:49.280644+0000","last_clean_scrub_stamp":"2023-10-16T08:39:49.280644+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:51:41.348858+0000","scrub_duration":0.0036633019999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"22'3","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:56.656761+0000","last_change":"2023-10-16T08:39:56.656883+0000","last_active":"2023-10-16T08:39:56.656761+0000","last_peered":"2023-10-16T08:39:56.656761+0000","last_clean":"2023-10-16T08:39:56.656761+0000","last_became_active":"2023-10-16T08:38:44.646042+0000","last_became_peered":"2023-10-16T08:38:44.646042+0000","last_unstale":"2023-10-16T08:39:56.656761+0000","last_undegraded":"2023-10-16T08:39:56.656761+0000","last_fullsized":"2023-10-16T08:39:56.656761+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2023-10-16T08:39:56.656531+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2023-10-16T08:39:56.656531+0000","last_clean_scrub_stamp":"2023-10-16T08:39:56.656531+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:31:21.757461+0000","scrub_duration":0.0076556489999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:42.365480+0000","last_change":"2023-10-16T08:40:42.365580+0000","last_active":"2023-10-16T08:40:42.365480+0000","last_peered":"2023-10-16T08:40:42.365480+0000","last_clean":"2023-10-16T08:40:42.365480+0000","last_became_active":"2023-10-16T08:38:45.670415+0000","last_became_peered":"2023-10-16T08:38:45.670415+0000","last_unstale":"2023-10-16T08:40:42.365480+0000","last_undegraded":"2023-10-16T08:40:42.365480+0000","last_fullsized":"2023-10-16T08:40:42.365480+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:42.365263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:42.365263+0000","last_clean_scrub_stamp":"2023-10-16T08:40:42.365263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:22:33.344640+0000","scrub_duration":0.003258103,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"22'2","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:01.687399+0000","last_change":"2023-10-16T08:40:01.687511+0000","last_active":"2023-10-16T08:40:01.687399+0000","last_peered":"2023-10-16T08:40:01.687399+0000","last_clean":"2023-10-16T08:40:01.687399+0000","last_became_active":"2023-10-16T08:38:44.655260+0000","last_became_peered":"2023-10-16T08:38:44.655260+0000","last_unstale":"2023-10-16T08:40:01.687399+0000","last_undegraded":"2023-10-16T08:40:01.687399+0000","last_fullsized":"2023-10-16T08:40:01.687399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'2","last_scrub_stamp":"2023-10-16T08:40:01.687151+0000","last_deep_scrub":"22'2","last_deep_scrub_stamp":"2023-10-16T08:40:01.687151+0000","last_clean_scrub_stamp":"2023-10-16T08:40:01.687151+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:32:11.992732+0000","scrub_duration":0.0081170519999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:06.489937+0000","last_change":"2023-10-16T08:40:06.490039+0000","last_active":"2023-10-16T08:40:06.489937+0000","last_peered":"2023-10-16T08:40:06.489937+0000","last_clean":"2023-10-16T08:40:06.489937+0000","last_became_active":"2023-10-16T08:38:45.667627+0000","last_became_peered":"2023-10-16T08:38:45.667627+0000","last_unstale":"2023-10-16T08:40:06.489937+0000","last_undegraded":"2023-10-16T08:40:06.489937+0000","last_fullsized":"2023-10-16T08:40:06.489937+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:06.489710+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:06.489710+0000","last_clean_scrub_stamp":"2023-10-16T08:40:06.489710+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:10:38.568870+0000","scrub_duration":0.003795746,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:51.623054+0000","last_change":"2023-10-16T08:39:51.623164+0000","last_active":"2023-10-16T08:39:51.623054+0000","last_peered":"2023-10-16T08:39:51.623054+0000","last_clean":"2023-10-16T08:39:51.623054+0000","last_became_active":"2023-10-16T08:38:44.661736+0000","last_became_peered":"2023-10-16T08:38:44.661736+0000","last_unstale":"2023-10-16T08:39:51.623054+0000","last_undegraded":"2023-10-16T08:39:51.623054+0000","last_fullsized":"2023-10-16T08:39:51.623054+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:51.622816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:51.622816+0000","last_clean_scrub_stamp":"2023-10-16T08:39:51.622816+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:32:29.535131+0000","scrub_duration":0.0038193530000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:32.401674+0000","last_change":"2023-10-16T08:40:32.401764+0000","last_active":"2023-10-16T08:40:32.401674+0000","last_peered":"2023-10-16T08:40:32.401674+0000","last_clean":"2023-10-16T08:40:32.401674+0000","last_became_active":"2023-10-16T08:38:45.666390+0000","last_became_peered":"2023-10-16T08:38:45.666390+0000","last_unstale":"2023-10-16T08:40:32.401674+0000","last_undegraded":"2023-10-16T08:40:32.401674+0000","last_fullsized":"2023-10-16T08:40:32.401674+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:32.401461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:32.401461+0000","last_clean_scrub_stamp":"2023-10-16T08:40:32.401461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:42:43.782091+0000","scrub_duration":0.003391678,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'119","reported_seq":179,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:50.308644+0000","last_change":"2023-10-16T08:39:50.308644+0000","last_active":"2023-10-16T08:39:50.308644+0000","last_peered":"2023-10-16T08:39:50.308644+0000","last_clean":"2023-10-16T08:39:50.308644+0000","last_became_active":"2023-10-16T08:38:34.310944+0000","last_became_peered":"2023-10-16T08:38:34.310944+0000","last_unstale":"2023-10-16T08:39:50.308644+0000","last_undegraded":"2023-10-16T08:39:50.308644+0000","last_fullsized":"2023-10-16T08:39:50.308644+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'119","last_scrub_stamp":"2023-10-16T08:39:50.308327+0000","last_deep_scrub":"22'119","last_deep_scrub_stamp":"2023-10-16T08:39:50.308327+0000","last_clean_scrub_stamp":"2023-10-16T08:39:50.308327+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:27:14.065843+0000","scrub_duration":0.014557508,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:48.490807+0000","last_change":"2023-10-16T08:39:48.490904+0000","last_active":"2023-10-16T08:39:48.490807+0000","last_peered":"2023-10-16T08:39:48.490807+0000","last_clean":"2023-10-16T08:39:48.490807+0000","last_became_active":"2023-10-16T08:38:44.654277+0000","last_became_peered":"2023-10-16T08:38:44.654277+0000","last_unstale":"2023-10-16T08:39:48.490807+0000","last_undegraded":"2023-10-16T08:39:48.490807+0000","last_fullsized":"2023-10-16T08:39:48.490807+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:48.490591+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:48.490591+0000","last_clean_scrub_stamp":"2023-10-16T08:39:48.490591+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T15:51:40.558806+0000","scrub_duration":0.0031485229999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:16.723373+0000","last_change":"2023-10-16T08:40:16.723476+0000","last_active":"2023-10-16T08:40:16.723373+0000","last_peered":"2023-10-16T08:40:16.723373+0000","last_clean":"2023-10-16T08:40:16.723373+0000","last_became_active":"2023-10-16T08:38:45.679409+0000","last_became_peered":"2023-10-16T08:38:45.679409+0000","last_unstale":"2023-10-16T08:40:16.723373+0000","last_undegraded":"2023-10-16T08:40:16.723373+0000","last_fullsized":"2023-10-16T08:40:16.723373+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:16.723108+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:16.723108+0000","last_clean_scrub_stamp":"2023-10-16T08:40:16.723108+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:46:41.397936+0000","scrub_duration":0.0042941769999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":24,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:55.336630+0000","last_change":"2023-10-16T08:39:55.336630+0000","last_active":"2023-10-16T08:39:55.336630+0000","last_peered":"2023-10-16T08:39:55.336630+0000","last_clean":"2023-10-16T08:39:55.336630+0000","last_became_active":"2023-10-16T08:38:44.651534+0000","last_became_peered":"2023-10-16T08:38:44.651534+0000","last_unstale":"2023-10-16T08:39:55.336630+0000","last_undegraded":"2023-10-16T08:39:55.336630+0000","last_fullsized":"2023-10-16T08:39:55.336630+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:55.336229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:55.336229+0000","last_clean_scrub_stamp":"2023-10-16T08:39:55.336229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:37:12.644946+0000","scrub_duration":0.0081414639999999993,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:52.170208+0000","last_change":"2023-10-16T08:39:52.170378+0000","last_active":"2023-10-16T08:39:52.170208+0000","last_peered":"2023-10-16T08:39:52.170208+0000","last_clean":"2023-10-16T08:39:52.170208+0000","last_became_active":"2023-10-16T08:38:45.650715+0000","last_became_peered":"2023-10-16T08:38:45.650715+0000","last_unstale":"2023-10-16T08:39:52.170208+0000","last_undegraded":"2023-10-16T08:39:52.170208+0000","last_fullsized":"2023-10-16T08:39:52.170208+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:52.169935+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:52.169935+0000","last_clean_scrub_stamp":"2023-10-16T08:39:52.169935+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:48:51.865234+0000","scrub_duration":0.0041537630000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:53.623737+0000","last_change":"2023-10-16T08:39:53.623864+0000","last_active":"2023-10-16T08:39:53.623737+0000","last_peered":"2023-10-16T08:39:53.623737+0000","last_clean":"2023-10-16T08:39:53.623737+0000","last_became_active":"2023-10-16T08:38:44.645451+0000","last_became_peered":"2023-10-16T08:38:44.645451+0000","last_unstale":"2023-10-16T08:39:53.623737+0000","last_undegraded":"2023-10-16T08:39:53.623737+0000","last_fullsized":"2023-10-16T08:39:53.623737+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:53.623513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:53.623513+0000","last_clean_scrub_stamp":"2023-10-16T08:39:53.623513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:58.932498+0000","scrub_duration":0.0035529649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:28.794322+0000","last_change":"2023-10-16T08:40:28.794322+0000","last_active":"2023-10-16T08:40:28.794322+0000","last_peered":"2023-10-16T08:40:28.794322+0000","last_clean":"2023-10-16T08:40:28.794322+0000","last_became_active":"2023-10-16T08:38:45.640765+0000","last_became_peered":"2023-10-16T08:38:45.640765+0000","last_unstale":"2023-10-16T08:40:28.794322+0000","last_undegraded":"2023-10-16T08:40:28.794322+0000","last_fullsized":"2023-10-16T08:40:28.794322+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:28.794008+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:28.794008+0000","last_clean_scrub_stamp":"2023-10-16T08:40:28.794008+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T17:08:40.882444+0000","scrub_duration":0.0036498640000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:53.528625+0000","last_change":"2023-10-16T08:39:53.528786+0000","last_active":"2023-10-16T08:39:53.528625+0000","last_peered":"2023-10-16T08:39:53.528625+0000","last_clean":"2023-10-16T08:39:53.528625+0000","last_became_active":"2023-10-16T08:38:44.654622+0000","last_became_peered":"2023-10-16T08:38:44.654622+0000","last_unstale":"2023-10-16T08:39:53.528625+0000","last_undegraded":"2023-10-16T08:39:53.528625+0000","last_fullsized":"2023-10-16T08:39:53.528625+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:53.528393+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:53.528393+0000","last_clean_scrub_stamp":"2023-10-16T08:39:53.528393+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:27.379794+0000","scrub_duration":0.0041697289999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:53.788332+0000","last_change":"2023-10-16T08:39:53.788477+0000","last_active":"2023-10-16T08:39:53.788332+0000","last_peered":"2023-10-16T08:39:53.788332+0000","last_clean":"2023-10-16T08:39:53.788332+0000","last_became_active":"2023-10-16T08:38:45.658594+0000","last_became_peered":"2023-10-16T08:38:45.658594+0000","last_unstale":"2023-10-16T08:39:53.788332+0000","last_undegraded":"2023-10-16T08:39:53.788332+0000","last_fullsized":"2023-10-16T08:39:53.788332+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:53.788113+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:53.788113+0000","last_clean_scrub_stamp":"2023-10-16T08:39:53.788113+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:43:02.143993+0000","scrub_duration":0.0036952420000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:45.752930+0000","last_change":"2023-10-16T08:39:45.752966+0000","last_active":"2023-10-16T08:39:45.752930+0000","last_peered":"2023-10-16T08:39:45.752930+0000","last_clean":"2023-10-16T08:39:45.752930+0000","last_became_active":"2023-10-16T08:38:44.633854+0000","last_became_peered":"2023-10-16T08:38:44.633854+0000","last_unstale":"2023-10-16T08:39:45.752930+0000","last_undegraded":"2023-10-16T08:39:45.752930+0000","last_fullsized":"2023-10-16T08:39:45.752930+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:45.752829+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:45.752829+0000","last_clean_scrub_stamp":"2023-10-16T08:39:45.752829+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T11:18:59.589626+0000","scrub_duration":0.0015287009999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":29,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:23.561166+0000","last_change":"2023-10-16T08:40:23.561262+0000","last_active":"2023-10-16T08:40:23.561166+0000","last_peered":"2023-10-16T08:40:23.561166+0000","last_clean":"2023-10-16T08:40:23.561166+0000","last_became_active":"2023-10-16T08:39:27.015266+0000","last_became_peered":"2023-10-16T08:39:27.015266+0000","last_unstale":"2023-10-16T08:40:23.561166+0000","last_undegraded":"2023-10-16T08:40:23.561166+0000","last_fullsized":"2023-10-16T08:40:23.561166+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:23.560948+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:23.560948+0000","last_clean_scrub_stamp":"2023-10-16T08:40:23.560948+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:00:17.028005+0000","scrub_duration":0.0038406479999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:04.332824+0000","last_change":"2023-10-16T08:40:04.332989+0000","last_active":"2023-10-16T08:40:04.332824+0000","last_peered":"2023-10-16T08:40:04.332824+0000","last_clean":"2023-10-16T08:40:04.332824+0000","last_became_active":"2023-10-16T08:38:44.655001+0000","last_became_peered":"2023-10-16T08:38:44.655001+0000","last_unstale":"2023-10-16T08:40:04.332824+0000","last_undegraded":"2023-10-16T08:40:04.332824+0000","last_fullsized":"2023-10-16T08:40:04.332824+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:04.332585+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:04.332585+0000","last_clean_scrub_stamp":"2023-10-16T08:40:04.332585+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:46:29.007413+0000","scrub_duration":0.003821592,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:37.806827+0000","last_change":"2023-10-16T08:40:37.807014+0000","last_active":"2023-10-16T08:40:37.806827+0000","last_peered":"2023-10-16T08:40:37.806827+0000","last_clean":"2023-10-16T08:40:37.806827+0000","last_became_active":"2023-10-16T08:38:45.638702+0000","last_became_peered":"2023-10-16T08:38:45.638702+0000","last_unstale":"2023-10-16T08:40:37.806827+0000","last_undegraded":"2023-10-16T08:40:37.806827+0000","last_fullsized":"2023-10-16T08:40:37.806827+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:37.806578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:37.806578+0000","last_clean_scrub_stamp":"2023-10-16T08:40:37.806578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:11:09.885738+0000","scrub_duration":0.0041059369999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"22'3","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:06.680541+0000","last_change":"2023-10-16T08:40:06.680647+0000","last_active":"2023-10-16T08:40:06.680541+0000","last_peered":"2023-10-16T08:40:06.680541+0000","last_clean":"2023-10-16T08:40:06.680541+0000","last_became_active":"2023-10-16T08:38:44.648312+0000","last_became_peered":"2023-10-16T08:38:44.648312+0000","last_unstale":"2023-10-16T08:40:06.680541+0000","last_undegraded":"2023-10-16T08:40:06.680541+0000","last_fullsized":"2023-10-16T08:40:06.680541+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2023-10-16T08:40:06.680326+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2023-10-16T08:40:06.680326+0000","last_clean_scrub_stamp":"2023-10-16T08:40:06.680326+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:42:24.980087+0000","scrub_duration":0.0047609419999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:27.612182+0000","last_change":"2023-10-16T08:40:27.612282+0000","last_active":"2023-10-16T08:40:27.612182+0000","last_peered":"2023-10-16T08:40:27.612182+0000","last_clean":"2023-10-16T08:40:27.612182+0000","last_became_active":"2023-10-16T08:38:45.662442+0000","last_became_peered":"2023-10-16T08:38:45.662442+0000","last_unstale":"2023-10-16T08:40:27.612182+0000","last_undegraded":"2023-10-16T08:40:27.612182+0000","last_fullsized":"2023-10-16T08:40:27.612182+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:27.611937+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:27.611937+0000","last_clean_scrub_stamp":"2023-10-16T08:40:27.611937+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T20:09:24.469054+0000","scrub_duration":0.0040286890000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"22'3","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:48.214304+0000","last_change":"2023-10-16T08:39:48.214423+0000","last_active":"2023-10-16T08:39:48.214304+0000","last_peered":"2023-10-16T08:39:48.214304+0000","last_clean":"2023-10-16T08:39:48.214304+0000","last_became_active":"2023-10-16T08:38:44.648155+0000","last_became_peered":"2023-10-16T08:38:44.648155+0000","last_unstale":"2023-10-16T08:39:48.214304+0000","last_undegraded":"2023-10-16T08:39:48.214304+0000","last_fullsized":"2023-10-16T08:39:48.214304+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2023-10-16T08:39:48.214061+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2023-10-16T08:39:48.214061+0000","last_clean_scrub_stamp":"2023-10-16T08:39:48.214061+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:48:53.523046+0000","scrub_duration":0.0072707400000000004,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:27.792442+0000","last_change":"2023-10-16T08:40:27.792545+0000","last_active":"2023-10-16T08:40:27.792442+0000","last_peered":"2023-10-16T08:40:27.792442+0000","last_clean":"2023-10-16T08:40:27.792442+0000","last_became_active":"2023-10-16T08:38:45.636751+0000","last_became_peered":"2023-10-16T08:38:45.636751+0000","last_unstale":"2023-10-16T08:40:27.792442+0000","last_undegraded":"2023-10-16T08:40:27.792442+0000","last_fullsized":"2023-10-16T08:40:27.792442+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:27.792223+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:27.792223+0000","last_clean_scrub_stamp":"2023-10-16T08:40:27.792223+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:07:51.515524+0000","scrub_duration":0.0034292670000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:51.303765+0000","last_change":"2023-10-16T08:39:51.303866+0000","last_active":"2023-10-16T08:39:51.303765+0000","last_peered":"2023-10-16T08:39:51.303765+0000","last_clean":"2023-10-16T08:39:51.303765+0000","last_became_active":"2023-10-16T08:38:44.646990+0000","last_became_peered":"2023-10-16T08:38:44.646990+0000","last_unstale":"2023-10-16T08:39:51.303765+0000","last_undegraded":"2023-10-16T08:39:51.303765+0000","last_fullsized":"2023-10-16T08:39:51.303765+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:51.303518+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:51.303518+0000","last_clean_scrub_stamp":"2023-10-16T08:39:51.303518+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:55:30.823021+0000","scrub_duration":0.0039969649999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:48.807872+0000","last_change":"2023-10-16T08:39:48.807982+0000","last_active":"2023-10-16T08:39:48.807872+0000","last_peered":"2023-10-16T08:39:48.807872+0000","last_clean":"2023-10-16T08:39:48.807872+0000","last_became_active":"2023-10-16T08:38:45.647449+0000","last_became_peered":"2023-10-16T08:38:45.647449+0000","last_unstale":"2023-10-16T08:39:48.807872+0000","last_undegraded":"2023-10-16T08:39:48.807872+0000","last_fullsized":"2023-10-16T08:39:48.807872+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:48.807616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:48.807616+0000","last_clean_scrub_stamp":"2023-10-16T08:39:48.807616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:23:04.229531+0000","scrub_duration":0.0038025490000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:48.714882+0000","last_change":"2023-10-16T08:39:48.714951+0000","last_active":"2023-10-16T08:39:48.714882+0000","last_peered":"2023-10-16T08:39:48.714882+0000","last_clean":"2023-10-16T08:39:48.714882+0000","last_became_active":"2023-10-16T08:38:44.630451+0000","last_became_peered":"2023-10-16T08:38:44.630451+0000","last_unstale":"2023-10-16T08:39:48.714882+0000","last_undegraded":"2023-10-16T08:39:48.714882+0000","last_fullsized":"2023-10-16T08:39:48.714882+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:39:48.714761+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:39:48.714761+0000","last_clean_scrub_stamp":"2023-10-16T08:39:48.714761+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:42:57.070641+0000","scrub_duration":0.0033819409999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:13.680916+0000","last_change":"2023-10-16T08:40:13.681012+0000","last_active":"2023-10-16T08:40:13.680916+0000","last_peered":"2023-10-16T08:40:13.680916+0000","last_clean":"2023-10-16T08:40:13.680916+0000","last_became_active":"2023-10-16T08:38:45.663100+0000","last_became_peered":"2023-10-16T08:38:45.663100+0000","last_unstale":"2023-10-16T08:40:13.680916+0000","last_undegraded":"2023-10-16T08:40:13.680916+0000","last_fullsized":"2023-10-16T08:40:13.680916+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:13.680679+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:13.680679+0000","last_clean_scrub_stamp":"2023-10-16T08:40:13.680679+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:24.518729+0000","scrub_duration":0.0037025669999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:10.638658+0000","last_change":"2023-10-16T08:40:10.638842+0000","last_active":"2023-10-16T08:40:10.638658+0000","last_peered":"2023-10-16T08:40:10.638658+0000","last_clean":"2023-10-16T08:40:10.638658+0000","last_became_active":"2023-10-16T08:38:44.646580+0000","last_became_peered":"2023-10-16T08:38:44.646580+0000","last_unstale":"2023-10-16T08:40:10.638658+0000","last_undegraded":"2023-10-16T08:40:10.638658+0000","last_fullsized":"2023-10-16T08:40:10.638658+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:10.638415+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:10.638415+0000","last_clean_scrub_stamp":"2023-10-16T08:40:10.638415+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:50:05.087112+0000","scrub_duration":0.003785058,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:12.671707+0000","last_change":"2023-10-16T08:40:12.671827+0000","last_active":"2023-10-16T08:40:12.671707+0000","last_peered":"2023-10-16T08:40:12.671707+0000","last_clean":"2023-10-16T08:40:12.671707+0000","last_became_active":"2023-10-16T08:38:45.662226+0000","last_became_peered":"2023-10-16T08:38:45.662226+0000","last_unstale":"2023-10-16T08:40:12.671707+0000","last_undegraded":"2023-10-16T08:40:12.671707+0000","last_fullsized":"2023-10-16T08:40:12.671707+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:12.671469+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:12.671469+0000","last_clean_scrub_stamp":"2023-10-16T08:40:12.671469+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:13:45.857171+0000","scrub_duration":0.003468933,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"22'3","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:53.387291+0000","last_change":"2023-10-16T08:39:53.387482+0000","last_active":"2023-10-16T08:39:53.387291+0000","last_peered":"2023-10-16T08:39:53.387291+0000","last_clean":"2023-10-16T08:39:53.387291+0000","last_became_active":"2023-10-16T08:38:44.659976+0000","last_became_peered":"2023-10-16T08:38:44.659976+0000","last_unstale":"2023-10-16T08:39:53.387291+0000","last_undegraded":"2023-10-16T08:39:53.387291+0000","last_fullsized":"2023-10-16T08:39:53.387291+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2023-10-16T08:39:53.387049+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2023-10-16T08:39:53.387049+0000","last_clean_scrub_stamp":"2023-10-16T08:39:53.387049+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T14:47:27.238450+0000","scrub_duration":0.0071616939999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:08.969126+0000","last_change":"2023-10-16T08:40:08.969225+0000","last_active":"2023-10-16T08:40:08.969126+0000","last_peered":"2023-10-16T08:40:08.969126+0000","last_clean":"2023-10-16T08:40:08.969126+0000","last_became_active":"2023-10-16T08:38:45.664616+0000","last_became_peered":"2023-10-16T08:38:45.664616+0000","last_unstale":"2023-10-16T08:40:08.969126+0000","last_undegraded":"2023-10-16T08:40:08.969126+0000","last_fullsized":"2023-10-16T08:40:08.969126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:08.968899+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:08.968899+0000","last_clean_scrub_stamp":"2023-10-16T08:40:08.968899+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T08:50:24.351118+0000","scrub_duration":0.0033892219999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:13.566725+0000","last_change":"2023-10-16T08:40:13.566817+0000","last_active":"2023-10-16T08:40:13.566725+0000","last_peered":"2023-10-16T08:40:13.566725+0000","last_clean":"2023-10-16T08:40:13.566725+0000","last_became_active":"2023-10-16T08:38:45.680231+0000","last_became_peered":"2023-10-16T08:38:45.680231+0000","last_unstale":"2023-10-16T08:40:13.566725+0000","last_undegraded":"2023-10-16T08:40:13.566725+0000","last_fullsized":"2023-10-16T08:40:13.566725+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:13.566499+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:13.566499+0000","last_clean_scrub_stamp":"2023-10-16T08:40:13.566499+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:36:54.787429+0000","scrub_duration":0.0033674909999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"22'3","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:58.403119+0000","last_change":"2023-10-16T08:39:58.403247+0000","last_active":"2023-10-16T08:39:58.403119+0000","last_peered":"2023-10-16T08:39:58.403119+0000","last_clean":"2023-10-16T08:39:58.403119+0000","last_became_active":"2023-10-16T08:38:44.663674+0000","last_became_peered":"2023-10-16T08:38:44.663674+0000","last_unstale":"2023-10-16T08:39:58.403119+0000","last_undegraded":"2023-10-16T08:39:58.403119+0000","last_fullsized":"2023-10-16T08:39:58.403119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2023-10-16T08:39:58.402923+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2023-10-16T08:39:58.402923+0000","last_clean_scrub_stamp":"2023-10-16T08:39:58.402923+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:13:31.588625+0000","scrub_duration":0.0069404790000000003,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:08.562697+0000","last_change":"2023-10-16T08:40:08.562797+0000","last_active":"2023-10-16T08:40:08.562697+0000","last_peered":"2023-10-16T08:40:08.562697+0000","last_clean":"2023-10-16T08:40:08.562697+0000","last_became_active":"2023-10-16T08:38:45.680266+0000","last_became_peered":"2023-10-16T08:38:45.680266+0000","last_unstale":"2023-10-16T08:40:08.562697+0000","last_undegraded":"2023-10-16T08:40:08.562697+0000","last_fullsized":"2023-10-16T08:40:08.562697+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:08.562471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:08.562471+0000","last_clean_scrub_stamp":"2023-10-16T08:40:08.562471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T10:29:33.941649+0000","scrub_duration":0.0036526010000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"22'3","reported_seq":26,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:01.344970+0000","last_change":"2023-10-16T08:40:01.345161+0000","last_active":"2023-10-16T08:40:01.344970+0000","last_peered":"2023-10-16T08:40:01.344970+0000","last_clean":"2023-10-16T08:40:01.344970+0000","last_became_active":"2023-10-16T08:38:44.652658+0000","last_became_peered":"2023-10-16T08:38:44.652658+0000","last_unstale":"2023-10-16T08:40:01.344970+0000","last_undegraded":"2023-10-16T08:40:01.344970+0000","last_fullsized":"2023-10-16T08:40:01.344970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'3","last_scrub_stamp":"2023-10-16T08:40:01.344742+0000","last_deep_scrub":"22'3","last_deep_scrub_stamp":"2023-10-16T08:40:01.344742+0000","last_clean_scrub_stamp":"2023-10-16T08:40:01.344742+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T19:07:12.182792+0000","scrub_duration":0.0070571119999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":22,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:24.819126+0000","last_change":"2023-10-16T08:40:24.819221+0000","last_active":"2023-10-16T08:40:24.819126+0000","last_peered":"2023-10-16T08:40:24.819126+0000","last_clean":"2023-10-16T08:40:24.819126+0000","last_became_active":"2023-10-16T08:38:45.664274+0000","last_became_peered":"2023-10-16T08:38:45.664274+0000","last_unstale":"2023-10-16T08:40:24.819126+0000","last_undegraded":"2023-10-16T08:40:24.819126+0000","last_fullsized":"2023-10-16T08:40:24.819126+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:24.818872+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:24.818872+0000","last_clean_scrub_stamp":"2023-10-16T08:40:24.818872+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:13:58.004574+0000","scrub_duration":0.0037996509999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"19'1","reported_seq":25,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:56.005280+0000","last_change":"2023-10-16T08:39:56.005280+0000","last_active":"2023-10-16T08:39:56.005280+0000","last_peered":"2023-10-16T08:39:56.005280+0000","last_clean":"2023-10-16T08:39:56.005280+0000","last_became_active":"2023-10-16T08:38:44.645000+0000","last_became_peered":"2023-10-16T08:38:44.645000+0000","last_unstale":"2023-10-16T08:39:56.005280+0000","last_undegraded":"2023-10-16T08:39:56.005280+0000","last_fullsized":"2023-10-16T08:39:56.005280+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"19'1","last_scrub_stamp":"2023-10-16T08:39:56.004937+0000","last_deep_scrub":"19'1","last_deep_scrub_stamp":"2023-10-16T08:39:56.004937+0000","last_clean_scrub_stamp":"2023-10-16T08:39:56.004937+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:42:14.304698+0000","scrub_duration":0.0037060180000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":23,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:40:20.647673+0000","last_change":"2023-10-16T08:40:20.647673+0000","last_active":"2023-10-16T08:40:20.647673+0000","last_peered":"2023-10-16T08:40:20.647673+0000","last_clean":"2023-10-16T08:40:20.647673+0000","last_became_active":"2023-10-16T08:38:45.663538+0000","last_became_peered":"2023-10-16T08:38:45.663538+0000","last_unstale":"2023-10-16T08:40:20.647673+0000","last_undegraded":"2023-10-16T08:40:20.647673+0000","last_fullsized":"2023-10-16T08:40:20.647673+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-16T08:40:20.647257+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-16T08:40:20.647257+0000","last_clean_scrub_stamp":"2023-10-16T08:40:20.647257+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T12:30:20.728617+0000","scrub_duration":0.0041945580000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"22'6","reported_seq":29,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-16T08:39:47.192216+0000","last_change":"2023-10-16T08:39:47.192267+0000","last_active":"2023-10-16T08:39:47.192216+0000","last_peered":"2023-10-16T08:39:47.192216+0000","last_clean":"2023-10-16T08:39:47.192216+0000","last_became_active":"2023-10-16T08:38:44.645854+0000","last_became_peered":"2023-10-16T08:38:44.645854+0000","last_unstale":"2023-10-16T08:39:47.192216+0000","last_undegraded":"2023-10-16T08:39:47.192216+0000","last_fullsized":"2023-10-16T08:39:47.192216+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"22'6","last_scrub_stamp":"2023-10-16T08:39:47.192138+0000","last_deep_scrub":"22'6","last_deep_scrub_stamp":"2023-10-16T08:39:47.192138+0000","last_clean_scrub_stamp":"2023-10-16T08:39:47.192138+0000","objects_scrubbed":2,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-17T09:31:12.293068+0000","scrub_duration":0.0072348439999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6065,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":61,"num_write_kb":28,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":5000,"num_omap_keys":10,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":12130,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":72,"ondisk_log_size":72,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574878,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27824,"kb_used_data":808,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344016,"statfs":{"total":96636764160,"available":96608272384,"internally_reserved":0,"allocated":827392,"data_stored":674262,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1589,"internal_metadata":27654603},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.875,"15min":0.85499999999999998},"min":{"1min":0.53500000000000003,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":1.224},{"interface":"front","average":{"1min":1.083,"5min":1.018,"15min":1.0069999999999999},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.3220000000000001,"5min":1.381,"15min":1.381},"last":0.998}]},{"osd":1,"last update":"Mon Oct 16 08:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.88300000000000001,"5min":0.92600000000000005,"15min":0.93300000000000005},"min":{"1min":0.61599999999999999,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":1.0720000000000001},{"interface":"front","average":{"1min":0.88500000000000001,"5min":0.76600000000000001,"15min":0.746},"min":{"1min":0.56000000000000005,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.1419999999999999},"last":0.78700000000000003}]},{"osd":2,"last update":"Mon Oct 16 08:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.81599999999999995,"15min":0.80400000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.351,"5min":1.351,"15min":1.351},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.878,"15min":0.87},"min":{"1min":0.68700000000000006,"5min":0.62,"15min":0.62},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.65700000000000003}]},{"osd":3,"last update":"Mon Oct 16 08:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.89600000000000002,"5min":0.86799999999999999,"15min":0.86299999999999999},"min":{"1min":0.59499999999999997,"5min":0.58299999999999996,"15min":0.58299999999999996},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":1.028},{"interface":"front","average":{"1min":1.0700000000000001,"5min":1.02,"15min":1.012},"min":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"max":{"1min":1.363,"5min":1.4099999999999999,"15min":1.4099999999999999},"last":0.90800000000000003}]},{"osd":4,"last update":"Mon Oct 16 08:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.86599999999999999,"15min":0.871},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":1.145},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.028,"15min":1.032},"min":{"1min":0.754,"5min":0.754,"15min":0.754},"max":{"1min":1.228,"5min":1.228,"15min":1.228},"last":0.95999999999999996}]},{"osd":5,"last update":"Mon Oct 16 08:40:32 2023","interfaces":[{"interface":"back","average":{"1min":0.99199999999999999,"5min":1.1000000000000001,"15min":1.1180000000000001},"min":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"max":{"1min":1.282,"5min":1.3600000000000001,"15min":1.3600000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":1.101,"5min":1.0649999999999999,"15min":1.0589999999999999},"min":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"max":{"1min":1.4019999999999999,"5min":1.4019999999999999,"15min":1.4019999999999999},"last":1.278}]},{"osd":6,"last update":"Mon Oct 16 08:40:32 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":1.0049999999999999,"15min":0.995},"min":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":1.3220000000000001},{"interface":"front","average":{"1min":0.98099999999999998,"5min":1.008,"15min":1.012},"min":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":1.103}]}]},{"osd":6,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":88579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:40:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0609999999999999,"5min":0.88600000000000001,"15min":0.85699999999999998},"min":{"1min":0.754,"5min":0.497,"15min":0.497},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.94499999999999995},{"interface":"front","average":{"1min":1.109,"5min":1.03,"15min":1.0169999999999999},"min":{"1min":0.83599999999999997,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.29,"5min":2.1899999999999999,"15min":2.1899999999999999},"last":1.163}]},{"osd":1,"last update":"Mon Oct 16 08:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.95499999999999996,"5min":0.95099999999999996,"15min":0.95099999999999996},"min":{"1min":0.65100000000000002,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.2050000000000001,"5min":1.306,"15min":1.306},"last":0.872},{"interface":"front","average":{"1min":1.0049999999999999,"5min":1.0149999999999999,"15min":1.0169999999999999},"min":{"1min":0.74199999999999999,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":1.1919999999999999,"5min":2.282,"15min":2.282},"last":0.97099999999999997}]},{"osd":2,"last update":"Mon Oct 16 08:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.96199999999999997,"5min":0.96799999999999997,"15min":0.96899999999999997},"min":{"1min":0.63600000000000001,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":1.351,"5min":1.4530000000000001,"15min":1.4530000000000001},"last":1.0649999999999999},{"interface":"front","average":{"1min":0.94599999999999995,"5min":0.86099999999999999,"15min":0.84699999999999998},"min":{"1min":0.79700000000000004,"5min":0.55800000000000005,"15min":0.55800000000000005},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.72699999999999998}]},{"osd":3,"last update":"Mon Oct 16 08:40:33 2023","interfaces":[{"interface":"back","average":{"1min":1.0209999999999999,"5min":0.95699999999999996,"15min":0.94599999999999995},"min":{"1min":0.73699999999999999,"5min":0.69199999999999995,"15min":0.69199999999999995},"max":{"1min":1.27,"5min":1.784,"15min":1.784},"last":1.21},{"interface":"front","average":{"1min":1.014,"5min":1.002,"15min":1},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"max":{"1min":1.292,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.81799999999999995}]},{"osd":4,"last update":"Mon Oct 16 08:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":1.014,"15min":1.0449999999999999},"min":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"max":{"1min":1.034,"5min":1.5600000000000001,"15min":1.5600000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":1.004,"5min":1.089,"15min":1.103},"min":{"1min":0.68700000000000006,"5min":0.68700000000000006,"15min":0.68700000000000006},"max":{"1min":1.4299999999999999,"5min":1.925,"15min":1.925},"last":0.996}]},{"osd":5,"last update":"Mon Oct 16 08:40:33 2023","interfaces":[{"interface":"back","average":{"1min":0.91100000000000003,"5min":0.98999999999999999,"15min":1.0029999999999999},"min":{"1min":0.65400000000000003,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.1200000000000001,"5min":1.6240000000000001,"15min":1.6240000000000001},"last":0.79400000000000004},{"interface":"front","average":{"1min":1.109,"5min":1.177,"15min":1.1890000000000001},"min":{"1min":0.68100000000000005,"5min":0.58699999999999997,"15min":0.58699999999999997},"max":{"1min":1.3560000000000001,"5min":2.1150000000000002,"15min":2.1150000000000002},"last":1.1140000000000001}]},{"osd":7,"last update":"Mon Oct 16 08:40:33 2023","interfaces":[{"interface":"back","average":{"1min":1.1990000000000001,"5min":1.2370000000000001,"15min":1.2430000000000001},"min":{"1min":0.92200000000000004,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.3959999999999999,"5min":2.04,"15min":2.04},"last":0.84499999999999997},{"interface":"front","average":{"1min":1.054,"5min":1.1339999999999999,"15min":1.147},"min":{"1min":0.83599999999999997,"5min":0.629,"15min":0.629},"max":{"1min":1.2589999999999999,"5min":1.675,"15min":1.675},"last":0.90200000000000002}]}]},{"osd":5,"up_from":13,"seq":55834574880,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27244,"kb_used_data":228,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344596,"statfs":{"total":96636764160,"available":96608866304,"internally_reserved":0,"allocated":233472,"data_stored":83402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:40:36 2023","interfaces":[{"interface":"back","average":{"1min":0.88100000000000001,"5min":0.85499999999999998,"15min":0.85099999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.216,"5min":1.216,"15min":1.216},"last":1.0780000000000001},{"interface":"front","average":{"1min":0.90100000000000002,"5min":0.91000000000000003,"15min":0.91200000000000003},"min":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"max":{"1min":1.2649999999999999,"5min":1.5149999999999999,"15min":1.5149999999999999},"last":0.872}]},{"osd":1,"last update":"Mon Oct 16 08:40:36 2023","interfaces":[{"interface":"back","average":{"1min":0.93700000000000006,"5min":0.85299999999999998,"15min":0.83899999999999997},"min":{"1min":0.60699999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"max":{"1min":1.24,"5min":1.3149999999999999,"15min":1.3149999999999999},"last":0.99399999999999999},{"interface":"front","average":{"1min":0.90800000000000003,"5min":0.92400000000000004,"15min":0.92700000000000005},"min":{"1min":0.501,"5min":0.501,"15min":0.501},"max":{"1min":1.1439999999999999,"5min":1.27,"15min":1.27},"last":1.117}]},{"osd":2,"last update":"Mon Oct 16 08:40:36 2023","interfaces":[{"interface":"back","average":{"1min":0.94499999999999995,"5min":0.93799999999999994,"15min":0.93700000000000006},"min":{"1min":0.63200000000000001,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.3080000000000001,"5min":1.456,"15min":1.456},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.95599999999999996,"5min":0.93500000000000005,"15min":0.93100000000000005},"min":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.1870000000000001,"5min":1.5569999999999999,"15min":1.5569999999999999},"last":0.94299999999999995}]},{"osd":3,"last update":"Mon Oct 16 08:40:36 2023","interfaces":[{"interface":"back","average":{"1min":0.96699999999999997,"5min":0.93899999999999995,"15min":0.93400000000000005},"min":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"max":{"1min":1.202,"5min":1.202,"15min":1.202},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.075,"5min":1.04,"15min":1.034},"min":{"1min":0.73999999999999999,"5min":0.63300000000000001,"15min":0.63300000000000001},"max":{"1min":1.2729999999999999,"5min":2.0209999999999999,"15min":2.0209999999999999},"last":1.0189999999999999}]},{"osd":4,"last update":"Mon Oct 16 08:40:36 2023","interfaces":[{"interface":"back","average":{"1min":0.92100000000000004,"5min":0.93799999999999994,"15min":0.94099999999999995},"min":{"1min":0.60999999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.393,"5min":1.696,"15min":1.696},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.93799999999999994,"5min":0.96699999999999997,"15min":0.97199999999999998},"min":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.2250000000000001,"5min":1.26,"15min":1.26},"last":1.153}]},{"osd":6,"last update":"Mon Oct 16 08:40:36 2023","interfaces":[{"interface":"back","average":{"1min":0.98499999999999999,"5min":1.0600000000000001,"15min":1.0720000000000001},"min":{"1min":0.748,"5min":0.621,"15min":0.621},"max":{"1min":1.359,"5min":2.1040000000000001,"15min":2.1040000000000001},"last":1.175},{"interface":"front","average":{"1min":1.056,"5min":1.1279999999999999,"15min":1.1399999999999999},"min":{"1min":0.79100000000000004,"5min":0.63900000000000001,"15min":0.63900000000000001},"max":{"1min":1.3260000000000001,"5min":2.2200000000000002,"15min":2.2200000000000002},"last":0.83499999999999996}]},{"osd":7,"last update":"Mon Oct 16 08:40:36 2023","interfaces":[{"interface":"back","average":{"1min":1.069,"5min":1.1160000000000001,"15min":1.1240000000000001},"min":{"1min":0.68400000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":1.274,"5min":1.496,"15min":1.496},"last":0.77900000000000003},{"interface":"front","average":{"1min":1.1160000000000001,"5min":1.179,"15min":1.1890000000000001},"min":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"max":{"1min":1.333,"5min":2.181,"15min":2.181},"last":1.226}]}]},{"osd":4,"up_from":13,"seq":55834574879,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27252,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344588,"statfs":{"total":96636764160,"available":96608858112,"internally_reserved":0,"allocated":241664,"data_stored":83438,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.96599999999999997,"5min":0.95499999999999996,"15min":0.95299999999999996},"min":{"1min":0.67000000000000004,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.234,"5min":1.583,"15min":1.583},"last":0.83099999999999996},{"interface":"front","average":{"1min":1.036,"5min":0.94599999999999995,"15min":0.93100000000000005},"min":{"1min":0.54700000000000004,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2390000000000001,"5min":1.5900000000000001,"15min":1.5900000000000001},"last":0.96599999999999997}]},{"osd":1,"last update":"Mon Oct 16 08:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.96199999999999997,"15min":0.97399999999999998},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":1.2,"5min":1.716,"15min":1.716},"last":1.2},{"interface":"front","average":{"1min":0.873,"5min":0.89400000000000002,"15min":0.89800000000000002},"min":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":1.161,"5min":1.5049999999999999,"15min":1.5049999999999999},"last":0.65300000000000002}]},{"osd":2,"last update":"Mon Oct 16 08:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.93500000000000005,"5min":0.998,"15min":1.008},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.212,"5min":1.627,"15min":1.627},"last":1.1399999999999999},{"interface":"front","average":{"1min":0.998,"5min":1.0089999999999999,"15min":1.0109999999999999},"min":{"1min":0.75,"5min":0.66200000000000003,"15min":0.66200000000000003},"max":{"1min":1.286,"5min":1.476,"15min":1.476},"last":0.75}]},{"osd":3,"last update":"Mon Oct 16 08:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.877,"5min":0.86299999999999999,"15min":0.86099999999999999},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.996,"5min":1.0580000000000001,"15min":1.0680000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.2170000000000001,"5min":1.802,"15min":1.802},"last":1.034}]},{"osd":5,"last update":"Mon Oct 16 08:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.96299999999999997,"5min":1.012,"15min":1.02},"min":{"1min":0.65400000000000003,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.212,"5min":1.399,"15min":1.399},"last":0.93600000000000005},{"interface":"front","average":{"1min":1.0049999999999999,"5min":0.99299999999999999,"15min":0.99099999999999999},"min":{"1min":0.627,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.383,"5min":1.623,"15min":1.623},"last":0.89200000000000002}]},{"osd":6,"last update":"Mon Oct 16 08:40:38 2023","interfaces":[{"interface":"back","average":{"1min":0.96399999999999997,"5min":1.0389999999999999,"15min":1.0509999999999999},"min":{"1min":0.67500000000000004,"5min":0.499,"15min":0.499},"max":{"1min":1.319,"5min":1.742,"15min":1.742},"last":1.2969999999999999},{"interface":"front","average":{"1min":1.107,"5min":1.147,"15min":1.153},"min":{"1min":0.76900000000000002,"5min":0.60599999999999998,"15min":0.60599999999999998},"max":{"1min":1.3560000000000001,"5min":1.657,"15min":1.657},"last":1}]},{"osd":7,"last update":"Mon Oct 16 08:40:38 2023","interfaces":[{"interface":"back","average":{"1min":1.1060000000000001,"5min":1.0760000000000001,"15min":1.071},"min":{"1min":0.69399999999999995,"5min":0.433,"15min":0.433},"max":{"1min":1.4399999999999999,"5min":1.667,"15min":1.667},"last":1.0149999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.1950000000000001,"15min":1.2110000000000001},"min":{"1min":0.80800000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.331,"5min":2.153,"15min":2.153},"last":1.2490000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27256,"kb_used_data":240,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344584,"statfs":{"total":96636764160,"available":96608854016,"internally_reserved":0,"allocated":245760,"data_stored":88579,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1591,"internal_metadata":27654601},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.69799999999999995,"15min":0.68300000000000005},"min":{"1min":0.51600000000000001,"5min":0.311,"15min":0.311},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.75700000000000001,"15min":0.74399999999999999},"min":{"1min":0.59399999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.98099999999999998}]},{"osd":1,"last update":"Mon Oct 16 08:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.70399999999999996,"15min":0.71299999999999997},"min":{"1min":0.44600000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.99199999999999999,"5min":1.284,"15min":1.284},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.70599999999999996,"15min":0.69399999999999995},"min":{"1min":0.53000000000000003,"5min":0.307,"15min":0.307},"max":{"1min":1.099,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.93400000000000005}]},{"osd":2,"last update":"Mon Oct 16 08:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.72499999999999998,"15min":0.71699999999999997},"min":{"1min":0.56399999999999995,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":1.008},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.80700000000000005,"15min":0.80000000000000004},"min":{"1min":0.63400000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":1.0940000000000001,"5min":1.337,"15min":1.337},"last":1.153}]},{"osd":4,"last update":"Mon Oct 16 08:40:37 2023","interfaces":[{"interface":"back","average":{"1min":0.92900000000000005,"5min":0.91500000000000004,"15min":0.91300000000000003},"min":{"1min":0.66300000000000003,"5min":0.57799999999999996,"15min":0.57799999999999996},"max":{"1min":1.3180000000000001,"5min":1.3180000000000001,"15min":1.3180000000000001},"last":0.88},{"interface":"front","average":{"1min":0.96099999999999997,"5min":0.98299999999999998,"15min":0.98699999999999999},"min":{"1min":0.77200000000000002,"5min":0.63100000000000001,"15min":0.63100000000000001},"max":{"1min":1.1830000000000001,"5min":1.395,"15min":1.395},"last":1.2}]},{"osd":5,"last update":"Mon Oct 16 08:40:37 2023","interfaces":[{"interface":"back","average":{"1min":1.024,"5min":1.0109999999999999,"15min":1.0089999999999999},"min":{"1min":0.73999999999999999,"5min":0.67600000000000005,"15min":0.67600000000000005},"max":{"1min":1.3360000000000001,"5min":1.5489999999999999,"15min":1.5489999999999999},"last":1.28},{"interface":"front","average":{"1min":1.1100000000000001,"5min":1.0620000000000001,"15min":1.054},"min":{"1min":0.82799999999999996,"5min":0.53200000000000003,"15min":0.53200000000000003},"max":{"1min":1.375,"5min":1.5,"15min":1.5},"last":1.071}]},{"osd":6,"last update":"Mon Oct 16 08:40:37 2023","interfaces":[{"interface":"back","average":{"1min":1.042,"5min":1.02,"15min":1.016},"min":{"1min":0.76800000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.2989999999999999,"5min":1.6619999999999999,"15min":1.6619999999999999},"last":1.0469999999999999},{"interface":"front","average":{"1min":1.0880000000000001,"5min":1.1339999999999999,"15min":1.1419999999999999},"min":{"1min":0.83499999999999996,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.403,"5min":1.73,"15min":1.73},"last":1.109}]},{"osd":7,"last update":"Mon Oct 16 08:40:37 2023","interfaces":[{"interface":"back","average":{"1min":1.143,"5min":1.127,"15min":1.125},"min":{"1min":0.91400000000000003,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.3640000000000001,"5min":1.609,"15min":1.609},"last":1.236},{"interface":"front","average":{"1min":1.123,"5min":1.0980000000000001,"15min":1.0940000000000001},"min":{"1min":0.91000000000000003,"5min":0.70299999999999996,"15min":0.70299999999999996},"max":{"1min":1.3140000000000001,"5min":1.837,"15min":1.837},"last":1.0940000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27240,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344600,"statfs":{"total":96636764160,"available":96608870400,"internally_reserved":0,"allocated":229376,"data_stored":83312,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:40:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65700000000000003,"5min":0.59499999999999997,"15min":0.58499999999999996},"min":{"1min":0.42799999999999999,"5min":0.249,"15min":0.249},"max":{"1min":0.96699999999999997,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.70699999999999996,"15min":0.70299999999999996},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.0409999999999999,"5min":1.272,"15min":1.272},"last":0.71899999999999997}]},{"osd":1,"last update":"Mon Oct 16 08:40:34 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.70999999999999996,"15min":0.70799999999999996},"min":{"1min":0.39900000000000002,"5min":0.318,"15min":0.318},"max":{"1min":1.145,"5min":1.444,"15min":1.444},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.82699999999999996,"5min":0.71399999999999997,"15min":0.69499999999999995},"min":{"1min":0.32100000000000001,"5min":0.312,"15min":0.312},"max":{"1min":1.0960000000000001,"5min":1.111,"15min":1.111},"last":0.96399999999999997}]},{"osd":3,"last update":"Mon Oct 16 08:40:34 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.66000000000000003,"15min":0.64200000000000002},"min":{"1min":0.35399999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":1.1830000000000001,"5min":1.1830000000000001,"15min":1.1830000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.88,"5min":0.80500000000000005,"15min":0.79300000000000004},"min":{"1min":0.44800000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.421,"5min":1.9810000000000001,"15min":1.9810000000000001},"last":1.0589999999999999}]},{"osd":4,"last update":"Mon Oct 16 08:40:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0009999999999999,"5min":0.94799999999999995,"15min":0.93899999999999995},"min":{"1min":0.72199999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"max":{"1min":1.3460000000000001,"5min":1.6180000000000001,"15min":1.6180000000000001},"last":1.1830000000000001},{"interface":"front","average":{"1min":1.0169999999999999,"5min":0.94999999999999996,"15min":0.93899999999999995},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.2849999999999999,"5min":1.343,"15min":1.343},"last":1.3069999999999999}]},{"osd":5,"last update":"Mon Oct 16 08:40:34 2023","interfaces":[{"interface":"back","average":{"1min":1.054,"5min":0.91600000000000004,"15min":0.89300000000000002},"min":{"1min":0.61099999999999999,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":1.1399999999999999},{"interface":"front","average":{"1min":1.119,"5min":1.0529999999999999,"15min":1.042},"min":{"1min":0.77000000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"max":{"1min":1.3149999999999999,"5min":2.0670000000000002,"15min":2.0670000000000002},"last":1.244}]},{"osd":6,"last update":"Mon Oct 16 08:40:34 2023","interfaces":[{"interface":"back","average":{"1min":1.113,"5min":1.05,"15min":1.04},"min":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"max":{"1min":1.3660000000000001,"5min":1.3979999999999999,"15min":1.3979999999999999},"last":1.111},{"interface":"front","average":{"1min":1.093,"5min":1.012,"15min":0.998},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.4550000000000001,"5min":1.5,"15min":1.5},"last":1.3320000000000001}]},{"osd":7,"last update":"Mon Oct 16 08:40:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1759999999999999,"5min":1.079,"15min":1.0629999999999999},"min":{"1min":0.80900000000000005,"5min":0.622,"15min":0.622},"max":{"1min":1.452,"5min":1.544,"15min":1.544},"last":1.216},{"interface":"front","average":{"1min":1.1319999999999999,"5min":1.0640000000000001,"15min":1.052},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":1.383,"5min":1.748,"15min":1.748},"last":1.2689999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574878,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27248,"kb_used_data":232,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344592,"statfs":{"total":96636764160,"available":96608862208,"internally_reserved":0,"allocated":237568,"data_stored":83472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Mon Oct 16 08:40:34 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63200000000000001,"15min":0.63},"min":{"1min":0.51900000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.93300000000000005,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.65500000000000003,"15min":0.63900000000000001},"min":{"1min":0.35499999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.86899999999999999}]},{"osd":2,"last update":"Mon Oct 16 08:40:34 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.61099999999999999,"15min":0.59299999999999997},"min":{"1min":0.38900000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":1.026,"5min":1.026,"15min":1.026},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.76800000000000002,"15min":0.76400000000000001},"min":{"1min":0.55200000000000005,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.069,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":1.0109999999999999}]},{"osd":3,"last update":"Mon Oct 16 08:40:34 2023","interfaces":[{"interface":"back","average":{"1min":0.91300000000000003,"5min":0.85399999999999998,"15min":0.84399999999999997},"min":{"1min":0.47099999999999997,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":1.151,"5min":1.794,"15min":1.794},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.86099999999999999,"5min":0.77100000000000002,"15min":0.75600000000000001},"min":{"1min":0.49399999999999999,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.169,"5min":1.3140000000000001,"15min":1.3140000000000001},"last":1.27}]},{"osd":4,"last update":"Mon Oct 16 08:40:34 2023","interfaces":[{"interface":"back","average":{"1min":0.98699999999999999,"5min":1.0129999999999999,"15min":1.0169999999999999},"min":{"1min":0.76800000000000002,"5min":0.63800000000000001,"15min":0.63800000000000001},"max":{"1min":1.2310000000000001,"5min":2.27,"15min":2.27},"last":1.2090000000000001},{"interface":"front","average":{"1min":0.98499999999999999,"5min":0.91900000000000004,"15min":0.90800000000000003},"min":{"1min":0.64000000000000001,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.327,"5min":1.4510000000000001,"15min":1.4510000000000001},"last":0.78000000000000003}]},{"osd":5,"last update":"Mon Oct 16 08:40:34 2023","interfaces":[{"interface":"back","average":{"1min":1.0029999999999999,"5min":0.98299999999999998,"15min":0.97999999999999998},"min":{"1min":0.82999999999999996,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":1.236,"5min":1.8919999999999999,"15min":1.8919999999999999},"last":0.95599999999999996},{"interface":"front","average":{"1min":1.1830000000000001,"5min":1.087,"15min":1.071},"min":{"1min":0.79800000000000004,"5min":0.55900000000000005,"15min":0.55900000000000005},"max":{"1min":1.419,"5min":1.54,"15min":1.54},"last":1.175}]},{"osd":6,"last update":"Mon Oct 16 08:40:34 2023","interfaces":[{"interface":"back","average":{"1min":1.1259999999999999,"5min":1.075,"15min":1.0669999999999999},"min":{"1min":0.83699999999999997,"5min":0.54000000000000004,"15min":0.54000000000000004},"max":{"1min":1.3280000000000001,"5min":1.6830000000000001,"15min":1.6830000000000001},"last":1.087},{"interface":"front","average":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"min":{"1min":0.88500000000000001,"5min":0.67200000000000004,"15min":0.67200000000000004},"max":{"1min":1.3160000000000001,"5min":1.994,"15min":1.994},"last":1.0529999999999999}]},{"osd":7,"last update":"Mon Oct 16 08:40:34 2023","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.1519999999999999,"15min":1.143},"min":{"1min":0.75,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":1.524,"5min":2.1930000000000001,"15min":2.1930000000000001},"last":1.395},{"interface":"front","average":{"1min":1.1659999999999999,"5min":1.1499999999999999,"15min":1.147},"min":{"1min":0.88600000000000001,"5min":0.68899999999999995,"15min":0.68899999999999995},"max":{"1min":1.359,"5min":2.5449999999999999,"15min":2.5449999999999999},"last":1.3180000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574878,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27832,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344008,"statfs":{"total":96636764160,"available":96608264192,"internally_reserved":0,"allocated":835584,"data_stored":674318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Mon Oct 16 08:40:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.59399999999999997,"15min":0.58099999999999996},"min":{"1min":0.32100000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.85399999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.75,"5min":0.63100000000000001,"15min":0.61099999999999999},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":1.2210000000000001,"5min":1.2210000000000001,"15min":1.2210000000000001},"last":0.872}]},{"osd":2,"last update":"Mon Oct 16 08:40:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.626,"15min":0.61599999999999999},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.94399999999999995,"5min":1.421,"15min":1.421},"last":0.625},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.68999999999999995,"15min":0.66900000000000004},"min":{"1min":0.38400000000000001,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.1439999999999999,"5min":1.2769999999999999,"15min":1.2769999999999999},"last":1.0349999999999999}]},{"osd":3,"last update":"Mon Oct 16 08:40:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.67500000000000004,"15min":0.65800000000000003},"min":{"1min":0.36499999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0109999999999999,"5min":1.0109999999999999,"15min":1.0109999999999999},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.95099999999999996,"5min":0.79500000000000004,"15min":0.76900000000000002},"min":{"1min":0.34100000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":1.3169999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.82499999999999996}]},{"osd":4,"last update":"Mon Oct 16 08:40:35 2023","interfaces":[{"interface":"back","average":{"1min":0.995,"5min":0.90500000000000003,"15min":0.89000000000000001},"min":{"1min":0.57899999999999996,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":1.2909999999999999,"5min":1.298,"15min":1.298},"last":0.94399999999999995},{"interface":"front","average":{"1min":0.95499999999999996,"5min":0.97699999999999998,"15min":0.98099999999999998},"min":{"1min":0.745,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.177,"5min":1.3540000000000001,"15min":1.3540000000000001},"last":0.98799999999999999}]},{"osd":5,"last update":"Mon Oct 16 08:40:35 2023","interfaces":[{"interface":"back","average":{"1min":1.014,"5min":0.92700000000000005,"15min":0.91300000000000003},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"max":{"1min":1.4330000000000001,"5min":1.4330000000000001,"15min":1.4330000000000001},"last":1.224},{"interface":"front","average":{"1min":1.073,"5min":1.0189999999999999,"15min":1.01},"min":{"1min":0.77400000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"max":{"1min":1.3740000000000001,"5min":1.522,"15min":1.522},"last":1.0740000000000001}]},{"osd":6,"last update":"Mon Oct 16 08:40:35 2023","interfaces":[{"interface":"back","average":{"1min":1.0920000000000001,"5min":1.0109999999999999,"15min":0.997},"min":{"1min":0.79500000000000004,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.248,"5min":1.4690000000000001,"15min":1.4690000000000001},"last":1.1919999999999999},{"interface":"front","average":{"1min":1.1950000000000001,"5min":1.0740000000000001,"15min":1.054},"min":{"1min":0.75800000000000001,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":1.103}]},{"osd":7,"last update":"Mon Oct 16 08:40:35 2023","interfaces":[{"interface":"back","average":{"1min":1.1479999999999999,"5min":1.04,"15min":1.022},"min":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"max":{"1min":1.345,"5min":1.4910000000000001,"15min":1.4910000000000001},"last":1.161},{"interface":"front","average":{"1min":1.167,"5min":1.091,"15min":1.079},"min":{"1min":0.82199999999999995,"5min":0.68500000000000005,"15min":0.68500000000000005},"max":{"1min":1.423,"5min":1.619,"15min":1.619},"last":1.1359999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":5267,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-16T08:40:49.865 DEBUG:teuthology.orchestra.run.smithi040:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-16T08:40:50.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.017+0000 7f81a0acc640 1 Processor -- start 2023-10-16T08:40:50.020 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.017+0000 7f81a0acc640 1 -- start start 2023-10-16T08:40:50.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.017+0000 7f81a0acc640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f819c06dd70 0x7f819c0ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:50.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.017+0000 7f81a0acc640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f819c0cffd0 con 0x7f819c0ceb00 2023-10-16T08:40:50.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.017+0000 7f81a0acc640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f819c0cf310 con 0x7f819c15ad80 2023-10-16T08:40:50.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.017+0000 7f81a0acc640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f819c0cf5f0 con 0x7f819c06dd70 2023-10-16T08:40:50.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.017+0000 7f8193fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f819c06dd70 0x7f819c0ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:50.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.017+0000 7f8193fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f819c06dd70 0x7f819c0ce5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39836/0 (socket says 172.21.15.40:39836) 2023-10-16T08:40:50.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.017+0000 7f8193fff640 1 -- 172.21.15.40:0/2072820990 learned_addr learned my addr 172.21.15.40:0/2072820990 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:40:50.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.017+0000 7f8193fff640 1 -- 172.21.15.40:0/2072820990 >> v1:172.21.15.86:6789/0 conn(0x7f819c15ad80 legacy=0x7f819c06d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:40:50.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.017+0000 7f8193fff640 1 -- 172.21.15.40:0/2072820990 >> v1:172.21.15.40:6789/0 conn(0x7f819c0ceb00 legacy=0x7f819c0d0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:50.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.017+0000 7f8193fff640 1 -- 172.21.15.40:0/2072820990 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f819c0cf8d0 con 0x7f819c06dd70 2023-10-16T08:40:50.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f8193fff640 1 --2- 172.21.15.40:0/2072820990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f819c06dd70 0x7f819c0ce5c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8184009870 tx=0x7f818401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=446d9b1b5e0ef397 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:50.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f8199d9b640 1 -- 172.21.15.40:0/2072820990 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81840aa650 con 0x7f819c06dd70 2023-10-16T08:40:50.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f8199d9b640 1 -- 172.21.15.40:0/2072820990 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81840aa7b0 con 0x7f819c06dd70 2023-10-16T08:40:50.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f8199d9b640 1 -- 172.21.15.40:0/2072820990 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81840aab40 con 0x7f819c06dd70 2023-10-16T08:40:50.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f81a0acc640 1 -- 172.21.15.40:0/2072820990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f819c06dd70 msgr2=0x7f819c0ce5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:50.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f81a0acc640 1 --2- 172.21.15.40:0/2072820990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f819c06dd70 0x7f819c0ce5c0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f8184009870 tx=0x7f818401c4d0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.027 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f81a0acc640 1 -- 172.21.15.40:0/2072820990 shutdown_connections 2023-10-16T08:40:50.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f81a0acc640 1 --2- 172.21.15.40:0/2072820990 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f819c06dd70 0x7f819c0ce5c0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f81a0acc640 1 -- 172.21.15.40:0/2072820990 >> 172.21.15.40:0/2072820990 conn(0x7f819c071580 msgr2=0x7f819c071990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:50.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f81a0acc640 1 -- 172.21.15.40:0/2072820990 shutdown_connections 2023-10-16T08:40:50.028 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f81a0acc640 1 -- 172.21.15.40:0/2072820990 wait complete. 2023-10-16T08:40:50.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f81a0acc640 1 Processor -- start 2023-10-16T08:40:50.029 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f81a0acc640 1 -- start start 2023-10-16T08:40:50.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f81a0acc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f819c06dd70 0x7f819c06b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:50.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f81a0acc640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f819c0ceb00 0x7f819c06b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:50.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f81a0acc640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f819c15ad80 0x7f819c14a730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:50.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f81a0acc640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c14ffa0 con 0x7f819c0ceb00 2023-10-16T08:40:50.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f81a0acc640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f819c150260 con 0x7f819c06dd70 2023-10-16T08:40:50.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f81a0acc640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f819c150540 con 0x7f819c15ad80 2023-10-16T08:40:50.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f8193fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f819c0ceb00 0x7f819c06b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:50.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f8193fff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f819c0ceb00 0x7f819c06b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39844/0 (socket says 172.21.15.40:39844) 2023-10-16T08:40:50.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f8193fff640 1 -- 172.21.15.40:0/1053055343 learned_addr learned my addr 172.21.15.40:0/1053055343 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:40:50.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f819a59c640 1 --2- 172.21.15.40:0/1053055343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f819c06dd70 0x7f819c06b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:50.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f819ad9d640 1 --2- 172.21.15.40:0/1053055343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f819c15ad80 0x7f819c14a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:50.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f8193fff640 1 -- 172.21.15.40:0/1053055343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f819c15ad80 msgr2=0x7f819c14a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:50.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f8193fff640 1 --2- 172.21.15.40:0/1053055343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f819c15ad80 0x7f819c14a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f8193fff640 1 -- 172.21.15.40:0/1053055343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f819c06dd70 msgr2=0x7f819c06b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:50.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f8193fff640 1 --2- 172.21.15.40:0/1053055343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f819c06dd70 0x7f819c06b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f8193fff640 1 -- 172.21.15.40:0/1053055343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81840094d0 con 0x7f819c0ceb00 2023-10-16T08:40:50.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f819ad9d640 1 --2- 172.21.15.40:0/1053055343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f819c15ad80 0x7f819c14a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:40:50.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f8193fff640 1 --2- 172.21.15.40:0/1053055343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f819c0ceb00 0x7f819c06b980 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f81840ac040 tx=0x7f818401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:50.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f8192ffd640 1 -- 172.21.15.40:0/1053055343 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81840b3800 con 0x7f819c0ceb00 2023-10-16T08:40:50.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.021+0000 7f81a0acc640 1 -- 172.21.15.40:0/1053055343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f819c150820 con 0x7f819c0ceb00 2023-10-16T08:40:50.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.025+0000 7f8192ffd640 1 -- 172.21.15.40:0/1053055343 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81840b3980 con 0x7f819c0ceb00 2023-10-16T08:40:50.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.025+0000 7f81a0acc640 1 -- 172.21.15.40:0/1053055343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f819c14ac70 con 0x7f819c0ceb00 2023-10-16T08:40:50.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.025+0000 7f8192ffd640 1 -- 172.21.15.40:0/1053055343 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81840b3d30 con 0x7f819c0ceb00 2023-10-16T08:40:50.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.025+0000 7f81a0acc640 1 -- 172.21.15.40:0/1053055343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f819c04e9c0 con 0x7f819c0ceb00 2023-10-16T08:40:50.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.025+0000 7f8192ffd640 1 -- 172.21.15.40:0/1053055343 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f81840c6390 con 0x7f819c0ceb00 2023-10-16T08:40:50.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.025+0000 7f8192ffd640 1 --2- 172.21.15.40:0/1053055343 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f817c0664a0 0x7f817c068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:50.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.025+0000 7f8192ffd640 1 -- 172.21.15.40:0/1053055343 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f818411eb50 con 0x7f819c0ceb00 2023-10-16T08:40:50.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.025+0000 7f819a59c640 1 --2- 172.21.15.40:0/1053055343 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f817c0664a0 0x7f817c068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:50.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.029+0000 7f819a59c640 1 --2- 172.21.15.40:0/1053055343 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f817c0664a0 0x7f817c068960 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f818c00a750 tx=0x7f818c008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:50.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.037+0000 7f8192ffd640 1 -- 172.21.15.40:0/1053055343 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f818411ee60 con 0x7f819c0ceb00 2023-10-16T08:40:50.193 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.189+0000 7f81a0acc640 1 -- 172.21.15.40:0/1053055343 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f819c0cb990 con 0x7f819c0ceb00 2023-10-16T08:40:50.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.197+0000 7f8192ffd640 1 -- 172.21.15.40:0/1053055343 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v10)=0 v10) v1 ==== 123+0+0 (secure 0 0 0) 0x7f81840efab0 con 0x7f819c0ceb00 2023-10-16T08:40:50.204 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.197+0000 7f8192ffd640 1 -- 172.21.15.40:0/1053055343 <== mon.0 v2:172.21.15.40:3300/0 8 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f81840b3e90 con 0x7f819c0ceb00 2023-10-16T08:40:50.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.201+0000 7f81a0acc640 1 -- 172.21.15.40:0/1053055343 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f817c0664a0 msgr2=0x7f817c068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:50.205 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.201+0000 7f81a0acc640 1 --2- 172.21.15.40:0/1053055343 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f817c0664a0 0x7f817c068960 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f818c00a750 tx=0x7f818c008040 comp rx=0 tx=0).stop 2023-10-16T08:40:50.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.201+0000 7f81a0acc640 1 -- 172.21.15.40:0/1053055343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f819c0ceb00 msgr2=0x7f819c06b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:50.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.201+0000 7f81a0acc640 1 --2- 172.21.15.40:0/1053055343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f819c0ceb00 0x7f819c06b980 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f81840ac040 tx=0x7f818401ad10 comp rx=0 tx=0).stop 2023-10-16T08:40:50.206 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.201+0000 7f81a0acc640 1 -- 172.21.15.40:0/1053055343 shutdown_connections 2023-10-16T08:40:50.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.201+0000 7f81a0acc640 1 --2- 172.21.15.40:0/1053055343 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f819c15ad80 0x7f819c14a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.201+0000 7f81a0acc640 1 --2- 172.21.15.40:0/1053055343 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f817c0664a0 0x7f817c068960 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.201+0000 7f81a0acc640 1 --2- 172.21.15.40:0/1053055343 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f819c0ceb00 0x7f819c06b980 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.207 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.201+0000 7f81a0acc640 1 --2- 172.21.15.40:0/1053055343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f819c06dd70 0x7f819c06b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.201+0000 7f81a0acc640 1 -- 172.21.15.40:0/1053055343 >> 172.21.15.40:0/1053055343 conn(0x7f819c071580 msgr2=0x7f819c0cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:50.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.201+0000 7f81a0acc640 1 -- 172.21.15.40:0/1053055343 shutdown_connections 2023-10-16T08:40:50.208 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.201+0000 7f81a0acc640 1 -- 172.21.15.40:0/1053055343 wait complete. 2023-10-16T08:40:50.220 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-16T08:40:50.221 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:40:50.327 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.325+0000 7f4573c65640 1 Processor -- start 2023-10-16T08:40:50.383 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.325+0000 7f4573c65640 1 -- start start 2023-10-16T08:40:50.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.325+0000 7f4573c65640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f456c06dcd0 0x7f456c0cec20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:50.388 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.325+0000 7f4573c65640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f456c0cf160 0x7f456c065a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:50.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.325+0000 7f4573c65640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f456c15ac10 con 0x7f456c0cbf50 2023-10-16T08:40:50.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.325+0000 7f4573c65640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f456c0cf600 con 0x7f456c0cf160 2023-10-16T08:40:50.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.325+0000 7f4573c65640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456c070a20 con 0x7f456c06dcd0 2023-10-16T08:40:50.389 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.325+0000 7f4572202640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f456c0cbf50 0x7f456c0b1410 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46274/0 (socket says 172.21.15.40:46274) 2023-10-16T08:40:50.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.325+0000 7f4571200640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f456c0cf160 0x7f456c065a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:50.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.325+0000 7f4572202640 1 -- 172.21.15.40:0/1945918807 learned_addr learned my addr 172.21.15.40:0/1945918807 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:40:50.390 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.325+0000 7f4571a01640 1 --2- 172.21.15.40:0/1945918807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f456c06dcd0 0x7f456c0cec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:50.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.325+0000 7f4571200640 1 -- 172.21.15.40:0/1945918807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f456c06dcd0 msgr2=0x7f456c0cec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:40:50.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.325+0000 7f4571200640 1 --2- 172.21.15.40:0/1945918807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f456c06dcd0 0x7f456c0cec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.325+0000 7f4571200640 1 -- 172.21.15.40:0/1945918807 >> v1:172.21.15.40:6789/0 conn(0x7f456c0cbf50 legacy=0x7f456c0b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:50.391 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.325+0000 7f4571200640 1 -- 172.21.15.40:0/1945918807 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f456c070d00 con 0x7f456c0cf160 2023-10-16T08:40:50.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.325+0000 7f45709ff640 1 -- 172.21.15.40:0/1945918807 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1440970183 0 0) 0x7f456c15ac10 con 0x7f456c0cbf50 2023-10-16T08:40:50.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4571200640 1 --2- 172.21.15.40:0/1945918807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f456c0cf160 0x7f456c065a80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f45680097c0 tx=0x7f456801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c83b9f7463dd75b server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:50.392 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f45709ff640 1 -- 172.21.15.40:0/1945918807 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45680ab650 con 0x7f456c0cf160 2023-10-16T08:40:50.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f45709ff640 1 -- 172.21.15.40:0/1945918807 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f45680ab7b0 con 0x7f456c0cf160 2023-10-16T08:40:50.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f45709ff640 1 -- 172.21.15.40:0/1945918807 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45680abb60 con 0x7f456c0cf160 2023-10-16T08:40:50.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4573c65640 1 -- 172.21.15.40:0/1945918807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f456c0cf160 msgr2=0x7f456c065a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:50.393 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4573c65640 1 --2- 172.21.15.40:0/1945918807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f456c0cf160 0x7f456c065a80 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f45680097c0 tx=0x7f456801c450 comp rx=0 tx=0).stop 2023-10-16T08:40:50.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4573c65640 1 -- 172.21.15.40:0/1945918807 shutdown_connections 2023-10-16T08:40:50.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4573c65640 1 --2- 172.21.15.40:0/1945918807 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f456c0cf160 0x7f456c065a80 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4573c65640 1 --2- 172.21.15.40:0/1945918807 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f456c06dcd0 0x7f456c0cec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.394 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4573c65640 1 -- 172.21.15.40:0/1945918807 >> 172.21.15.40:0/1945918807 conn(0x7f456c0cc410 msgr2=0x7f456c0cc820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:50.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4573c65640 1 -- 172.21.15.40:0/1945918807 shutdown_connections 2023-10-16T08:40:50.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4573c65640 1 -- 172.21.15.40:0/1945918807 wait complete. 2023-10-16T08:40:50.395 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4573c65640 1 Processor -- start 2023-10-16T08:40:50.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4573c65640 1 -- start start 2023-10-16T08:40:50.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4573c65640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f456c06dcd0 0x7f456c0b1480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:50.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4573c65640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f456c0cbf50 0x7f456c14b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:50.396 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4573c65640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f456c0cf160 0x7f456c0b05b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:50.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4573c65640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456c152330 con 0x7f456c0cbf50 2023-10-16T08:40:50.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4573c65640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f456c14b8d0 con 0x7f456c0cf160 2023-10-16T08:40:50.397 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4573c65640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f456c0ae540 con 0x7f456c06dcd0 2023-10-16T08:40:50.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4571200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f456c0cbf50 0x7f456c14b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:50.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4571200640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f456c0cbf50 0x7f456c14b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39858/0 (socket says 172.21.15.40:39858) 2023-10-16T08:40:50.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4571200640 1 -- 172.21.15.40:0/935169831 learned_addr learned my addr 172.21.15.40:0/935169831 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:40:50.398 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4571200640 1 -- 172.21.15.40:0/935169831 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f456c06dcd0 msgr2=0x7f456c0b1480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:50.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4571200640 1 --2- 172.21.15.40:0/935169831 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f456c06dcd0 0x7f456c0b1480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4572202640 1 --2- 172.21.15.40:0/935169831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f456c0cf160 0x7f456c0b05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:50.399 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4571200640 1 -- 172.21.15.40:0/935169831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f456c0cf160 msgr2=0x7f456c0b05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:50.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4571200640 1 --2- 172.21.15.40:0/935169831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f456c0cf160 0x7f456c0b05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4571200640 1 -- 172.21.15.40:0/935169831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4568009420 con 0x7f456c0cbf50 2023-10-16T08:40:50.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f4571200640 1 --2- 172.21.15.40:0/935169831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f456c0cbf50 0x7f456c14b390 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f456801a080 tx=0x7f45680191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:50.400 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f455e7fc640 1 -- 172.21.15.40:0/935169831 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45680bf450 con 0x7f456c0cbf50 2023-10-16T08:40:50.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f455e7fc640 1 -- 172.21.15.40:0/935169831 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f45680bf5b0 con 0x7f456c0cbf50 2023-10-16T08:40:50.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.329+0000 7f455e7fc640 1 -- 172.21.15.40:0/935169831 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45680bf980 con 0x7f456c0cbf50 2023-10-16T08:40:50.401 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.333+0000 7f4573c65640 1 -- 172.21.15.40:0/935169831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f456c151ce0 con 0x7f456c0cbf50 2023-10-16T08:40:50.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.333+0000 7f4573c65640 1 -- 172.21.15.40:0/935169831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f456c0b0af0 con 0x7f456c0cbf50 2023-10-16T08:40:50.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.333+0000 7f4573c65640 1 -- 172.21.15.40:0/935169831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f456c151b40 con 0x7f456c0cbf50 2023-10-16T08:40:50.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.333+0000 7f455e7fc640 1 -- 172.21.15.40:0/935169831 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f45680bfae0 con 0x7f456c0cbf50 2023-10-16T08:40:50.402 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.333+0000 7f455e7fc640 1 --2- 172.21.15.40:0/935169831 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f45500664d0 0x7f4550068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:50.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.333+0000 7f455e7fc640 1 -- 172.21.15.40:0/935169831 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f456811ed00 con 0x7f456c0cbf50 2023-10-16T08:40:50.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.333+0000 7f4571a01640 1 --2- 172.21.15.40:0/935169831 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f45500664d0 0x7f4550068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:50.403 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.333+0000 7f4571a01640 1 --2- 172.21.15.40:0/935169831 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f45500664d0 0x7f4550068990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f45600070b0 tx=0x7f4560007040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:50.404 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.341+0000 7f455e7fc640 1 -- 172.21.15.40:0/935169831 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f45680ec160 con 0x7f456c0cbf50 2023-10-16T08:40:50.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.525+0000 7f4573c65640 1 -- 172.21.15.40:0/935169831 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f456c151b40 con 0x7f456c0cbf50 2023-10-16T08:40:50.529 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.525+0000 7f455e7fc640 1 -- 172.21.15.40:0/935169831 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4898 (secure 0 0 0) 0x7f456c151b40 con 0x7f456c0cbf50 2023-10-16T08:40:50.530 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:40:50.530 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/32529848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":32529848},{"type":"v1","addr":"172.21.15.86:6837","nonce":32529848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2044121591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2044121591},{"type":"v1","addr":"172.21.15.86:6836","nonce":2044121591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1275484997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1275484997},{"type":"v1","addr":"172.21.15.40:6833","nonce":1275484997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:38:44.984772+0000","modified":"2023-10-16T08:38:47.305255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/148507680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":148507680},{"type":"v1","addr":"172.21.15.40:6835","nonce":148507680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T08:40:50.531 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 6 2023-10-16T08:40:50.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.529+0000 7f4573c65640 1 -- 172.21.15.40:0/935169831 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f45500664d0 msgr2=0x7f4550068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:50.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.529+0000 7f4573c65640 1 --2- 172.21.15.40:0/935169831 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f45500664d0 0x7f4550068990 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f45600070b0 tx=0x7f4560007040 comp rx=0 tx=0).stop 2023-10-16T08:40:50.533 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.529+0000 7f4573c65640 1 -- 172.21.15.40:0/935169831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f456c0cbf50 msgr2=0x7f456c14b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:50.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.529+0000 7f4573c65640 1 --2- 172.21.15.40:0/935169831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f456c0cbf50 0x7f456c14b390 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f456801a080 tx=0x7f45680191f0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.529+0000 7f4573c65640 1 -- 172.21.15.40:0/935169831 shutdown_connections 2023-10-16T08:40:50.534 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.529+0000 7f4573c65640 1 --2- 172.21.15.40:0/935169831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f456c0cf160 0x7f456c0b05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.529+0000 7f4573c65640 1 --2- 172.21.15.40:0/935169831 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f45500664d0 0x7f4550068990 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.529+0000 7f4573c65640 1 --2- 172.21.15.40:0/935169831 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f456c0cbf50 0x7f456c14b390 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.535 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.529+0000 7f4573c65640 1 --2- 172.21.15.40:0/935169831 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f456c06dcd0 0x7f456c0b1480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.529+0000 7f4573c65640 1 -- 172.21.15.40:0/935169831 >> 172.21.15.40:0/935169831 conn(0x7f456c0cc410 msgr2=0x7f456c06d080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:50.536 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.529+0000 7f4573c65640 1 -- 172.21.15.40:0/935169831 shutdown_connections 2023-10-16T08:40:50.537 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.529+0000 7f4573c65640 1 -- 172.21.15.40:0/935169831 wait complete. 2023-10-16T08:40:50.549 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-16T08:40:50.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.669+0000 7f5a4dcd4640 1 Processor -- start 2023-10-16T08:40:50.675 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.669+0000 7f5a4dcd4640 1 -- start start 2023-10-16T08:40:50.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.673+0000 7f5a4dcd4640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5a480cd960 0x7f5a480cddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:50.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.673+0000 7f5a4dcd4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a480d0d40 0x7f5a480d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:50.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.673+0000 7f5a4dcd4640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5a480ce300 con 0x7f5a4815cce0 2023-10-16T08:40:50.677 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.673+0000 7f5a4dcd4640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5a480b7f80 con 0x7f5a480cd960 2023-10-16T08:40:50.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.673+0000 7f5a4dcd4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a480b7370 con 0x7f5a480d0d40 2023-10-16T08:40:50.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.673+0000 7f5a3ffff640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f5a480cd960 0x7f5a480cddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:50.678 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.673+0000 7f5a477fe640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7f5a4815cce0 0x7f5a480b68d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.40:35818/0 (socket says 172.21.15.40:35818) 2023-10-16T08:40:50.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.673+0000 7f5a477fe640 1 -- 172.21.15.40:0/977743037 learned_addr learned my addr 172.21.15.40:0/977743037 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:40:50.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.673+0000 7f5a47fff640 1 --2- 172.21.15.40:0/977743037 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a480d0d40 0x7f5a480d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:50.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.673+0000 7f5a46ffd640 1 -- 172.21.15.40:0/977743037 <== mon.1 v1:172.21.15.86:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1567023935 0 0) 0x7f5a480ce300 con 0x7f5a4815cce0 2023-10-16T08:40:50.679 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.673+0000 7f5a46ffd640 1 -- 172.21.15.40:0/977743037 --> v1:172.21.15.86:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5a34003530 con 0x7f5a4815cce0 2023-10-16T08:40:50.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.673+0000 7f5a47fff640 1 -- 172.21.15.40:0/977743037 >> v2:172.21.15.86:3300/0 conn(0x7f5a480cd960 msgr2=0x7f5a480cddc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:40:50.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.673+0000 7f5a47fff640 1 --2- 172.21.15.40:0/977743037 >> v2:172.21.15.86:3300/0 conn(0x7f5a480cd960 0x7f5a480cddc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.673+0000 7f5a47fff640 1 -- 172.21.15.40:0/977743037 >> v1:172.21.15.86:6789/0 conn(0x7f5a4815cce0 legacy=0x7f5a480b68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:50.680 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.673+0000 7f5a47fff640 1 -- 172.21.15.40:0/977743037 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a480b7650 con 0x7f5a480d0d40 2023-10-16T08:40:50.681 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.673+0000 7f5a3ffff640 1 --2- 172.21.15.40:0/977743037 >> v2:172.21.15.86:3300/0 conn(0x7f5a480cd960 0x7f5a480cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-16T08:40:50.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.673+0000 7f5a47fff640 1 --2- 172.21.15.40:0/977743037 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a480d0d40 0x7f5a480d11d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009d90 tx=0x7f5a4001c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=383b922e3d06c9ec server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:50.684 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a46ffd640 1 -- 172.21.15.40:0/977743037 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a400a5900 con 0x7f5a480d0d40 2023-10-16T08:40:50.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a46ffd640 1 -- 172.21.15.40:0/977743037 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5a400a5a60 con 0x7f5a480d0d40 2023-10-16T08:40:50.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a46ffd640 1 -- 172.21.15.40:0/977743037 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a400a5e10 con 0x7f5a480d0d40 2023-10-16T08:40:50.685 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a4dcd4640 1 -- 172.21.15.40:0/977743037 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a480d0d40 msgr2=0x7f5a480d11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a4dcd4640 1 --2- 172.21.15.40:0/977743037 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a480d0d40 0x7f5a480d11d0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f5a40009d90 tx=0x7f5a4001c950 comp rx=0 tx=0).stop 2023-10-16T08:40:50.686 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a4dcd4640 1 -- 172.21.15.40:0/977743037 shutdown_connections 2023-10-16T08:40:50.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a4dcd4640 1 --2- 172.21.15.40:0/977743037 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a480d0d40 0x7f5a480d11d0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a4dcd4640 1 --2- 172.21.15.40:0/977743037 >> v2:172.21.15.86:3300/0 conn(0x7f5a480cd960 0x7f5a480cddc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a4dcd4640 1 -- 172.21.15.40:0/977743037 >> 172.21.15.40:0/977743037 conn(0x7f5a4806a290 msgr2=0x7f5a4806a6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:50.687 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a4dcd4640 1 -- 172.21.15.40:0/977743037 shutdown_connections 2023-10-16T08:40:50.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a4dcd4640 1 -- 172.21.15.40:0/977743037 wait complete. 2023-10-16T08:40:50.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a4dcd4640 1 Processor -- start 2023-10-16T08:40:50.688 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a4dcd4640 1 -- start start 2023-10-16T08:40:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a4dcd4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a480cd960 0x7f5a480c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a4dcd4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a480d0d40 0x7f5a481f5b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:50.689 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a4dcd4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a4815cce0 0x7f5a48157700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:50.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a4dcd4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a48147b70 con 0x7f5a480d0d40 2023-10-16T08:40:50.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a4dcd4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5a480c84a0 con 0x7f5a4815cce0 2023-10-16T08:40:50.690 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a4dcd4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5a48159bd0 con 0x7f5a480cd960 2023-10-16T08:40:50.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a3ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a480d0d40 0x7f5a481f5b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:50.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a3ffff640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a480d0d40 0x7f5a481f5b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39870/0 (socket says 172.21.15.40:39870) 2023-10-16T08:40:50.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a3ffff640 1 -- 172.21.15.40:0/1542556156 learned_addr learned my addr 172.21.15.40:0/1542556156 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:40:50.691 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a47fff640 1 --2- 172.21.15.40:0/1542556156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a4815cce0 0x7f5a48157700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:50.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a477fe640 1 --2- 172.21.15.40:0/1542556156 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a480cd960 0x7f5a480c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:50.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a3ffff640 1 -- 172.21.15.40:0/1542556156 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a480cd960 msgr2=0x7f5a480c7f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:50.692 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a3ffff640 1 --2- 172.21.15.40:0/1542556156 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a480cd960 0x7f5a480c7f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a3ffff640 1 -- 172.21.15.40:0/1542556156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a4815cce0 msgr2=0x7f5a48157700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:50.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a3ffff640 1 --2- 172.21.15.40:0/1542556156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a4815cce0 0x7f5a48157700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.693 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a3ffff640 1 -- 172.21.15.40:0/1542556156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a30009420 con 0x7f5a480d0d40 2023-10-16T08:40:50.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a477fe640 1 --2- 172.21.15.40:0/1542556156 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a480cd960 0x7f5a480c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:40:50.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a3ffff640 1 --2- 172.21.15.40:0/1542556156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a480d0d40 0x7f5a481f5b00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000ebe0 tx=0x7f5a3000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:50.694 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a44ff9640 1 -- 172.21.15.40:0/1542556156 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a3000eee0 con 0x7f5a480d0d40 2023-10-16T08:40:50.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a44ff9640 1 -- 172.21.15.40:0/1542556156 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f5a3000cc30 con 0x7f5a480d0d40 2023-10-16T08:40:50.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a4dcd4640 1 -- 172.21.15.40:0/1542556156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5a40009a70 con 0x7f5a480d0d40 2023-10-16T08:40:50.695 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a44ff9640 1 -- 172.21.15.40:0/1542556156 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a30013470 con 0x7f5a480d0d40 2023-10-16T08:40:50.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.677+0000 7f5a4dcd4640 1 -- 172.21.15.40:0/1542556156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5a48159060 con 0x7f5a480d0d40 2023-10-16T08:40:50.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.681+0000 7f5a4dcd4640 1 -- 172.21.15.40:0/1542556156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5a48159d60 con 0x7f5a480d0d40 2023-10-16T08:40:50.696 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.681+0000 7f5a44ff9640 1 -- 172.21.15.40:0/1542556156 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f5a300135d0 con 0x7f5a480d0d40 2023-10-16T08:40:50.697 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.681+0000 7f5a44ff9640 1 --2- 172.21.15.40:0/1542556156 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f5a280664a0 0x7f5a28068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:50.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.681+0000 7f5a44ff9640 1 -- 172.21.15.40:0/1542556156 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f5a300840c0 con 0x7f5a480d0d40 2023-10-16T08:40:50.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.681+0000 7f5a477fe640 1 --2- 172.21.15.40:0/1542556156 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f5a280664a0 0x7f5a28068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:50.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.681+0000 7f5a477fe640 1 --2- 172.21.15.40:0/1542556156 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f5a280664a0 0x7f5a28068960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5a38005890 tx=0x7f5a38009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:50.698 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.685+0000 7f5a44ff9640 1 -- 172.21.15.40:0/1542556156 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f5a30051420 con 0x7f5a480d0d40 2023-10-16T08:40:50.879 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.877+0000 7f5a4dcd4640 1 -- 172.21.15.40:0/1542556156 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5a48159000 con 0x7f5a480d0d40 2023-10-16T08:40:50.880 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.877+0000 7f5a44ff9640 1 -- 172.21.15.40:0/1542556156 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4898 (secure 0 0 0) 0x7f5a30054a40 con 0x7f5a480d0d40 2023-10-16T08:40:50.881 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:40:50.881 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":6,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/32529848","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6835","nonce":32529848},{"type":"v1","addr":"172.21.15.86:6837","nonce":32529848}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":2},{"gid":4258,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6836/2044121591","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":2044121591},{"type":"v1","addr":"172.21.15.86:6836","nonce":2044121591}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3},{"gid":4278,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.40:6833/1275484997","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6832","nonce":1275484997},{"type":"v1","addr":"172.21.15.40:6833","nonce":1275484997}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":6,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-16T08:38:44.984772+0000","modified":"2023-10-16T08:38:47.305255+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}},"max_mds":1,"in":[0],"up":{"mds_0":4287},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4287":{"gid":4287,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.40:6835/148507680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6834","nonce":148507680},{"type":"v1","addr":"172.21.15.40:6835","nonce":148507680}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-10-16T08:40:50.882 INFO:teuthology.orchestra.run.smithi040.stderr:dumped fsmap epoch 6 2023-10-16T08:40:50.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.877+0000 7f5a4dcd4640 1 -- 172.21.15.40:0/1542556156 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f5a280664a0 msgr2=0x7f5a28068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:50.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.877+0000 7f5a4dcd4640 1 --2- 172.21.15.40:0/1542556156 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f5a280664a0 0x7f5a28068960 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f5a38005890 tx=0x7f5a38009040 comp rx=0 tx=0).stop 2023-10-16T08:40:50.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.877+0000 7f5a4dcd4640 1 -- 172.21.15.40:0/1542556156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a480d0d40 msgr2=0x7f5a481f5b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:50.883 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.877+0000 7f5a4dcd4640 1 --2- 172.21.15.40:0/1542556156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a480d0d40 0x7f5a481f5b00 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5a3000ebe0 tx=0x7f5a3000c520 comp rx=0 tx=0).stop 2023-10-16T08:40:50.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.877+0000 7f5a4dcd4640 1 -- 172.21.15.40:0/1542556156 shutdown_connections 2023-10-16T08:40:50.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.877+0000 7f5a4dcd4640 1 --2- 172.21.15.40:0/1542556156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a4815cce0 0x7f5a48157700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.884 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.877+0000 7f5a4dcd4640 1 --2- 172.21.15.40:0/1542556156 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f5a280664a0 0x7f5a28068960 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.877+0000 7f5a4dcd4640 1 --2- 172.21.15.40:0/1542556156 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f5a480d0d40 0x7f5a481f5b00 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.877+0000 7f5a4dcd4640 1 --2- 172.21.15.40:0/1542556156 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f5a480cd960 0x7f5a480c7f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:50.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.877+0000 7f5a4dcd4640 1 -- 172.21.15.40:0/1542556156 >> 172.21.15.40:0/1542556156 conn(0x7f5a4806a290 msgr2=0x7f5a4806a670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:50.885 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.877+0000 7f5a4dcd4640 1 -- 172.21.15.40:0/1542556156 shutdown_connections 2023-10-16T08:40:50.886 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:50.877+0000 7f5a4dcd4640 1 -- 172.21.15.40:0/1542556156 wait complete. 2023-10-16T08:40:50.900 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-16T08:40:51.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.013+0000 7f23bacf2640 1 Processor -- start 2023-10-16T08:40:51.019 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.013+0000 7f23bacf2640 1 -- start start 2023-10-16T08:40:51.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.017+0000 7f23bacf2640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f23b40cae50 0x7f23b406b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:51.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.017+0000 7f23bacf2640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7f23b40adec0 0x7f23b406bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:51.021 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.017+0000 7f23bacf2640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23b40adca0 con 0x7f23b415b040 2023-10-16T08:40:51.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.017+0000 7f23bacf2640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b40c5840 con 0x7f23b40cae50 2023-10-16T08:40:51.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.017+0000 7f23bacf2640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23b40cf890 con 0x7f23b40adec0 2023-10-16T08:40:51.022 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.017+0000 7f23b3fff640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f23b40cae50 0x7f23b406b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:51.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.017+0000 7f23b8a8e640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f23b415b040 0x7f23b40ca740 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46286/0 (socket says 172.21.15.40:46286) 2023-10-16T08:40:51.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.017+0000 7f23b8a8e640 1 -- 172.21.15.40:0/2248666058 learned_addr learned my addr 172.21.15.40:0/2248666058 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:40:51.023 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.017+0000 7f23b928f640 1 --2- 172.21.15.40:0/2248666058 >> v2:172.21.15.86:3300/0 conn(0x7f23b40adec0 0x7f23b406bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:51.024 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.017+0000 7f23b3fff640 1 -- 172.21.15.40:0/2248666058 >> v2:172.21.15.86:3300/0 conn(0x7f23b40adec0 msgr2=0x7f23b406bcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:40:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.017+0000 7f23b3fff640 1 --2- 172.21.15.40:0/2248666058 >> v2:172.21.15.86:3300/0 conn(0x7f23b40adec0 0x7f23b406bcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.025 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.017+0000 7f23b37fe640 1 -- 172.21.15.40:0/2248666058 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462770314 0 0) 0x7f23b40adca0 con 0x7f23b415b040 2023-10-16T08:40:51.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.017+0000 7f23b3fff640 1 -- 172.21.15.40:0/2248666058 >> v1:172.21.15.40:6789/0 conn(0x7f23b415b040 legacy=0x7f23b40ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:51.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.017+0000 7f23b3fff640 1 -- 172.21.15.40:0/2248666058 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23b40cf5b0 con 0x7f23b40cae50 2023-10-16T08:40:51.026 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.017+0000 7f23b928f640 1 --2- 172.21.15.40:0/2248666058 >> v2:172.21.15.86:3300/0 conn(0x7f23b40adec0 0x7f23b406bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-16T08:40:51.030 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.017+0000 7f23b3fff640 1 --2- 172.21.15.40:0/2248666058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23b40cae50 0x7f23b406b7b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f23b40cb390 tx=0x7f23a801c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4344130b47b6c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:51.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.017+0000 7f23b37fe640 1 -- 172.21.15.40:0/2248666058 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a80ab600 con 0x7f23b40cae50 2023-10-16T08:40:51.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.017+0000 7f23b37fe640 1 -- 172.21.15.40:0/2248666058 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f23a80a7050 con 0x7f23b40cae50 2023-10-16T08:40:51.031 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.017+0000 7f23b37fe640 1 -- 172.21.15.40:0/2248666058 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a80ab9a0 con 0x7f23b40cae50 2023-10-16T08:40:51.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.021+0000 7f23bacf2640 1 -- 172.21.15.40:0/2248666058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23b40cae50 msgr2=0x7f23b406b7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:51.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.021+0000 7f23bacf2640 1 --2- 172.21.15.40:0/2248666058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23b40cae50 0x7f23b406b7b0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f23b40cb390 tx=0x7f23a801c310 comp rx=0 tx=0).stop 2023-10-16T08:40:51.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.021+0000 7f23bacf2640 1 -- 172.21.15.40:0/2248666058 shutdown_connections 2023-10-16T08:40:51.032 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.021+0000 7f23bacf2640 1 --2- 172.21.15.40:0/2248666058 >> v2:172.21.15.86:3300/0 conn(0x7f23b40adec0 0x7f23b406bcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.021+0000 7f23bacf2640 1 --2- 172.21.15.40:0/2248666058 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23b40cae50 0x7f23b406b7b0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.021+0000 7f23bacf2640 1 -- 172.21.15.40:0/2248666058 >> 172.21.15.40:0/2248666058 conn(0x7f23b40ce960 msgr2=0x7f23b40ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:51.033 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.021+0000 7f23bacf2640 1 -- 172.21.15.40:0/2248666058 shutdown_connections 2023-10-16T08:40:51.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.021+0000 7f23bacf2640 1 -- 172.21.15.40:0/2248666058 wait complete. 2023-10-16T08:40:51.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.021+0000 7f23bacf2640 1 Processor -- start 2023-10-16T08:40:51.034 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.021+0000 7f23bacf2640 1 -- start start 2023-10-16T08:40:51.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.021+0000 7f23bacf2640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23b40adec0 0x7f23b40ce100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:51.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.021+0000 7f23bacf2640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23b40cae50 0x7f23b4152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:51.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.021+0000 7f23bacf2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23b415b040 0x7f23b414cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:51.035 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.021+0000 7f23bacf2640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b40c9110 con 0x7f23b40adec0 2023-10-16T08:40:51.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.021+0000 7f23bacf2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23b4152e90 con 0x7f23b415b040 2023-10-16T08:40:51.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.021+0000 7f23bacf2640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23b4153070 con 0x7f23b40cae50 2023-10-16T08:40:51.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.021+0000 7f23b8a8e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23b40adec0 0x7f23b40ce100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:51.036 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.021+0000 7f23b8a8e640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23b40adec0 0x7f23b40ce100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39890/0 (socket says 172.21.15.40:39890) 2023-10-16T08:40:51.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.021+0000 7f23b8a8e640 1 -- 172.21.15.40:0/1520573155 learned_addr learned my addr 172.21.15.40:0/1520573155 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:40:51.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.025+0000 7f23b8a8e640 1 -- 172.21.15.40:0/1520573155 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23b40cae50 msgr2=0x7f23b4152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:51.037 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.025+0000 7f23b8a8e640 1 --2- 172.21.15.40:0/1520573155 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23b40cae50 0x7f23b4152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.025+0000 7f23b8a8e640 1 -- 172.21.15.40:0/1520573155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23b415b040 msgr2=0x7f23b414cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:51.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.025+0000 7f23b8a8e640 1 --2- 172.21.15.40:0/1520573155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23b415b040 0x7f23b414cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.038 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.025+0000 7f23b8a8e640 1 -- 172.21.15.40:0/1520573155 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23a8009420 con 0x7f23b40adec0 2023-10-16T08:40:51.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.025+0000 7f23b8a8e640 1 --2- 172.21.15.40:0/1520573155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23b40adec0 0x7f23b40ce100 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f23a400c550 tx=0x7f23a400ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:51.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.025+0000 7f23b17fa640 1 -- 172.21.15.40:0/1520573155 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a40074e0 con 0x7f23b40adec0 2023-10-16T08:40:51.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.025+0000 7f23bacf2640 1 -- 172.21.15.40:0/1520573155 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23b4155060 con 0x7f23b40adec0 2023-10-16T08:40:51.039 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.025+0000 7f23b17fa640 1 -- 172.21.15.40:0/1520573155 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f23a4007640 con 0x7f23b40adec0 2023-10-16T08:40:51.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.025+0000 7f23b17fa640 1 -- 172.21.15.40:0/1520573155 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23a4004a10 con 0x7f23b40adec0 2023-10-16T08:40:51.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.025+0000 7f23bacf2640 1 -- 172.21.15.40:0/1520573155 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23b41555e0 con 0x7f23b40adec0 2023-10-16T08:40:51.040 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.025+0000 7f23bacf2640 1 -- 172.21.15.40:0/1520573155 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23b40c9250 con 0x7f23b40adec0 2023-10-16T08:40:51.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.025+0000 7f23b17fa640 1 -- 172.21.15.40:0/1520573155 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f23a4004b70 con 0x7f23b40adec0 2023-10-16T08:40:51.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.025+0000 7f23b17fa640 1 --2- 172.21.15.40:0/1520573155 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f23a00664a0 0x7f23a0068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:51.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.025+0000 7f23b17fa640 1 -- 172.21.15.40:0/1520573155 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7f23a40832a0 con 0x7f23b40adec0 2023-10-16T08:40:51.041 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.025+0000 7f23b3fff640 1 --2- 172.21.15.40:0/1520573155 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f23a00664a0 0x7f23a0068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:51.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.029+0000 7f23b3fff640 1 --2- 172.21.15.40:0/1520573155 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f23a00664a0 0x7f23a0068960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f23a80ad000 tx=0x7f23a8004740 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:51.042 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.029+0000 7f23b17fa640 1 -- 172.21.15.40:0/1520573155 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f23b40c9250 con 0x7f23b40adec0 2023-10-16T08:40:51.191 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.189+0000 7f23bacf2640 1 -- 172.21.15.40:0/1520573155 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f23b40c9250 con 0x7f23b40adec0 2023-10-16T08:40:51.194 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.189+0000 7f23b17fa640 1 -- 172.21.15.40:0/1520573155 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16741 (secure 0 0 0) 0x7f23a4050600 con 0x7f23b40adec0 2023-10-16T08:40:51.195 INFO:teuthology.orchestra.run.smithi040.stdout: 2023-10-16T08:40:51.195 INFO:teuthology.orchestra.run.smithi040.stdout:{"epoch":22,"fsid":"cc6c59f7-0179-4f54-b71d-89fa67221543","created":"2023-10-16T08:38:20.187908+0000","modified":"2023-10-16T08:39:26.948454+0000","last_up_change":"2023-10-16T08:38:32.212293+0000","last_in_change":"2023-10-16T08:38:22.620481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-16T08:38:33.014668+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-10-16T08:38:43.055215+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-10-16T08:38:43.971666+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b731cf12-d90b-4411-b989-7bd8fa503126","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6816","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6818","nonce":3244406559}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6820","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6822","nonce":3244406559}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6828","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6830","nonce":3244406559}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6824","nonce":3244406559},{"type":"v1","addr":"172.21.15.40:6826","nonce":3244406559}]},"public_addr":"172.21.15.40:6818/3244406559","cluster_addr":"172.21.15.40:6822/3244406559","heartbeat_back_addr":"172.21.15.40:6830/3244406559","heartbeat_front_addr":"172.21.15.40:6826/3244406559","state":["exists","up"]},{"osd":1,"uuid":"994268ab-dc28-42c8-8f2a-38eb2aedd884","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6817","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6819","nonce":2222522452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6821","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6823","nonce":2222522452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6829","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6831","nonce":2222522452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6825","nonce":2222522452},{"type":"v1","addr":"172.21.15.40:6827","nonce":2222522452}]},"public_addr":"172.21.15.40:6819/2222522452","cluster_addr":"172.21.15.40:6823/2222522452","heartbeat_back_addr":"172.21.15.40:6831/2222522452","heartbeat_front_addr":"172.21.15.40:6827/2222522452","state":["exists","up"]},{"osd":2,"uuid":"4b223048-c3fd-4bc5-8cf7-de6a0e2a8e32","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6800","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6801","nonce":2328483711}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6802","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6803","nonce":2328483711}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6806","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6807","nonce":2328483711}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6804","nonce":2328483711},{"type":"v1","addr":"172.21.15.40:6805","nonce":2328483711}]},"public_addr":"172.21.15.40:6801/2328483711","cluster_addr":"172.21.15.40:6803/2328483711","heartbeat_back_addr":"172.21.15.40:6807/2328483711","heartbeat_front_addr":"172.21.15.40:6805/2328483711","state":["exists","up"]},{"osd":3,"uuid":"f6ab290a-faf9-4699-93f1-0d2696a90b09","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6808","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6809","nonce":1155811160}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6810","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6811","nonce":1155811160}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6814","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6815","nonce":1155811160}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.40:6812","nonce":1155811160},{"type":"v1","addr":"172.21.15.40:6813","nonce":1155811160}]},"public_addr":"172.21.15.40:6809/1155811160","cluster_addr":"172.21.15.40:6811/1155811160","heartbeat_back_addr":"172.21.15.40:6815/1155811160","heartbeat_front_addr":"172.21.15.40:6813/1155811160","state":["exists","up"]},{"osd":4,"uuid":"89381ba9-8997-4473-a63f-c142d2231ece","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6809","nonce":150800249}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6811","nonce":150800249}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6819","nonce":150800249}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":150800249},{"type":"v1","addr":"172.21.15.86:6814","nonce":150800249}]},"public_addr":"172.21.15.86:6809/150800249","cluster_addr":"172.21.15.86:6811/150800249","heartbeat_back_addr":"172.21.15.86:6819/150800249","heartbeat_front_addr":"172.21.15.86:6814/150800249","state":["exists","up"]},{"osd":5,"uuid":"0467d942-124c-414c-b589-a79570b74d88","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6825","nonce":751032064}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6827","nonce":751032064}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6831","nonce":751032064}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":751032064},{"type":"v1","addr":"172.21.15.86:6829","nonce":751032064}]},"public_addr":"172.21.15.86:6825/751032064","cluster_addr":"172.21.15.86:6827/751032064","heartbeat_back_addr":"172.21.15.86:6831/751032064","heartbeat_front_addr":"172.21.15.86:6829/751032064","state":["exists","up"]},{"osd":6,"uuid":"3ffc263f-93b2-436d-8106-a45438435c39","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6813","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6815","nonce":2566952765}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6817","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6818","nonce":2566952765}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6823","nonce":2566952765}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":2566952765},{"type":"v1","addr":"172.21.15.86:6821","nonce":2566952765}]},"public_addr":"172.21.15.86:6815/2566952765","cluster_addr":"172.21.15.86:6818/2566952765","heartbeat_back_addr":"172.21.15.86:6823/2566952765","heartbeat_front_addr":"172.21.15.86:6821/2566952765","state":["exists","up"]},{"osd":7,"uuid":"0d7d34a3-a570-49fd-b542-25fd99bdf35b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6801","nonce":2560845614}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6803","nonce":2560845614}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6807","nonce":2560845614}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2560845614},{"type":"v1","addr":"172.21.15.86:6805","nonce":2560845614}]},"public_addr":"172.21.15.86:6801/2560845614","cluster_addr":"172.21.15.86:6803/2560845614","heartbeat_back_addr":"172.21.15.86:6807/2560845614","heartbeat_front_addr":"172.21.15.86:6805/2560845614","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.309239+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.268123+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.210853+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.221510+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.091679+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.197560+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.232202+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-10-16T08:38:28.306961+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.5","mappings":[{"from":3,"to":1}]},{"pgid":"3.21","mappings":[{"from":5,"to":1}]},{"pgid":"3.22","mappings":[{"from":5,"to":2}]},{"pgid":"3.2f","mappings":[{"from":5,"to":2}]},{"pgid":"3.3f","mappings":[{"from":7,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-16T08:40:51.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.193+0000 7f23bacf2640 1 -- 172.21.15.40:0/1520573155 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f23a00664a0 msgr2=0x7f23a0068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:51.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.193+0000 7f23bacf2640 1 --2- 172.21.15.40:0/1520573155 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f23a00664a0 0x7f23a0068960 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f23a80ad000 tx=0x7f23a8004740 comp rx=0 tx=0).stop 2023-10-16T08:40:51.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.193+0000 7f23bacf2640 1 -- 172.21.15.40:0/1520573155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23b40adec0 msgr2=0x7f23b40ce100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:51.197 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.193+0000 7f23bacf2640 1 --2- 172.21.15.40:0/1520573155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23b40adec0 0x7f23b40ce100 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f23a400c550 tx=0x7f23a400ca10 comp rx=0 tx=0).stop 2023-10-16T08:40:51.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.193+0000 7f23bacf2640 1 -- 172.21.15.40:0/1520573155 shutdown_connections 2023-10-16T08:40:51.198 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.193+0000 7f23bacf2640 1 --2- 172.21.15.40:0/1520573155 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f23b415b040 0x7f23b414cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.193+0000 7f23bacf2640 1 --2- 172.21.15.40:0/1520573155 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f23b40cae50 0x7f23b4152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.193+0000 7f23bacf2640 1 --2- 172.21.15.40:0/1520573155 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f23a00664a0 0x7f23a0068960 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.193+0000 7f23bacf2640 1 --2- 172.21.15.40:0/1520573155 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f23b40adec0 0x7f23b40ce100 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.199 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.193+0000 7f23bacf2640 1 -- 172.21.15.40:0/1520573155 >> 172.21.15.40:0/1520573155 conn(0x7f23b40ce960 msgr2=0x7f23b40afd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:51.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.193+0000 7f23bacf2640 1 -- 172.21.15.40:0/1520573155 shutdown_connections 2023-10-16T08:40:51.200 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.193+0000 7f23bacf2640 1 -- 172.21.15.40:0/1520573155 wait complete. 2023-10-16T08:40:51.215 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-16T08:40:51.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8cc43b640 1 Processor -- start 2023-10-16T08:40:51.341 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8cc43b640 1 -- start start 2023-10-16T08:40:51.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8cc43b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8c415b040 0x7fa8c40ca740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:51.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8cc43b640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7fa8c40adec0 0x7fa8c40cac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:51.344 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8cc43b640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8c40adca0 con 0x7fa8c40c8fb0 2023-10-16T08:40:51.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8cc43b640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8c40c5840 con 0x7fa8c40adec0 2023-10-16T08:40:51.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8cc43b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40cf890 con 0x7fa8c415b040 2023-10-16T08:40:51.345 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8ca9d8640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7fa8c40c8fb0 0x7fa8c40c9440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46298/0 (socket says 172.21.15.40:46298) 2023-10-16T08:40:51.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8ca9d8640 1 -- 172.21.15.40:0/2999128994 learned_addr learned my addr 172.21.15.40:0/2999128994 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:40:51.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8c99d6640 1 --2- 172.21.15.40:0/2999128994 >> v2:172.21.15.86:3300/0 conn(0x7fa8c40adec0 0x7fa8c40cac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:51.346 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8ca1d7640 1 --2- 172.21.15.40:0/2999128994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8c415b040 0x7fa8c40ca740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:51.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8c91d5640 1 -- 172.21.15.40:0/2999128994 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1111019349 0 0) 0x7fa8c40adca0 con 0x7fa8c40c8fb0 2023-10-16T08:40:51.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8c91d5640 1 -- 172.21.15.40:0/2999128994 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa8b0003530 con 0x7fa8c40c8fb0 2023-10-16T08:40:51.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8c91d5640 1 -- 172.21.15.40:0/2999128994 <== mon.0 v1:172.21.15.40:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2947988974 0 0) 0x7fa8b0003530 con 0x7fa8c40c8fb0 2023-10-16T08:40:51.347 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8c91d5640 1 -- 172.21.15.40:0/2999128994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8c415b040 msgr2=0x7fa8c40ca740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:40:51.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8c91d5640 1 --2- 172.21.15.40:0/2999128994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8c415b040 0x7fa8c40ca740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8c91d5640 1 -- 172.21.15.40:0/2999128994 >> v2:172.21.15.86:3300/0 conn(0x7fa8c40adec0 msgr2=0x7fa8c40cac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:40:51.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8c91d5640 1 --2- 172.21.15.40:0/2999128994 >> v2:172.21.15.86:3300/0 conn(0x7fa8c40adec0 0x7fa8c40cac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.348 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8c91d5640 1 -- 172.21.15.40:0/2999128994 --> v1:172.21.15.40:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c40cf5b0 con 0x7fa8c40c8fb0 2023-10-16T08:40:51.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8c91d5640 1 -- 172.21.15.40:0/2999128994 <== mon.0 v1:172.21.15.40:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2383167287 0 0) 0x7fa8bc001680 con 0x7fa8c40c8fb0 2023-10-16T08:40:51.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8c91d5640 1 -- 172.21.15.40:0/2999128994 >> v1:172.21.15.40:6789/0 conn(0x7fa8c40c8fb0 legacy=0x7fa8c40c9440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:51.349 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8c91d5640 1 --2- 172.21.15.40:0/2999128994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8b00039f0 0x7fa8b0003e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:51.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8c91d5640 1 --2- 172.21.15.40:0/2999128994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b00047d0 0x7fa8b0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:51.350 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8c91d5640 1 --2- 172.21.15.40:0/2999128994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8c415b040 0x7fa8b0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:51.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8c91d5640 1 -- 172.21.15.40:0/2999128994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40c5840 con 0x7fa8b00039f0 2023-10-16T08:40:51.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8c91d5640 1 -- 172.21.15.40:0/2999128994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40cf890 con 0x7fa8b00047d0 2023-10-16T08:40:51.351 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.337+0000 7fa8c91d5640 1 -- 172.21.15.40:0/2999128994 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b000ba90 con 0x7fa8c415b040 2023-10-16T08:40:51.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.341+0000 7fa8ca1d7640 1 --2- 172.21.15.40:0/2999128994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8b00039f0 0x7fa8b0003e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:51.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.341+0000 7fa8c99d6640 1 --2- 172.21.15.40:0/2999128994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b00047d0 0x7fa8b0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:51.352 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.341+0000 7fa8ca9d8640 1 --2- 172.21.15.40:0/2999128994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8c415b040 0x7fa8b0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:51.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.341+0000 7fa8ca1d7640 1 -- 172.21.15.40:0/2999128994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8c415b040 msgr2=0x7fa8b0005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:40:51.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.341+0000 7fa8ca1d7640 1 --2- 172.21.15.40:0/2999128994 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8c415b040 0x7fa8b0005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.341+0000 7fa8ca1d7640 1 -- 172.21.15.40:0/2999128994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b00047d0 msgr2=0x7fa8b0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:40:51.353 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.341+0000 7fa8ca1d7640 1 --2- 172.21.15.40:0/2999128994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b00047d0 0x7fa8b0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.354 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.341+0000 7fa8ca1d7640 1 -- 172.21.15.40:0/2999128994 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8b0003530 con 0x7fa8b00039f0 2023-10-16T08:40:51.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.341+0000 7fa8ca1d7640 1 --2- 172.21.15.40:0/2999128994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8b00039f0 0x7fa8b0003e10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009f90 tx=0x7fa8c001cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90385b550709450c server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:51.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.341+0000 7fa8c91d5640 1 -- 172.21.15.40:0/2999128994 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c00a5ad0 con 0x7fa8b00039f0 2023-10-16T08:40:51.357 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.341+0000 7fa8c91d5640 1 -- 172.21.15.40:0/2999128994 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fa8c000f040 con 0x7fa8b00039f0 2023-10-16T08:40:51.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.341+0000 7fa8c91d5640 1 -- 172.21.15.40:0/2999128994 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8c00a5e90 con 0x7fa8b00039f0 2023-10-16T08:40:51.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.341+0000 7fa8cc43b640 1 -- 172.21.15.40:0/2999128994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8b00039f0 msgr2=0x7fa8b0003e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:51.358 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.341+0000 7fa8cc43b640 1 --2- 172.21.15.40:0/2999128994 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8b00039f0 0x7fa8b0003e10 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa8c0009f90 tx=0x7fa8c001cbd0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.341+0000 7fa8ca9d8640 1 -- 172.21.15.40:0/2999128994 reap_dead start 2023-10-16T08:40:51.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.341+0000 7fa8cc43b640 1 -- 172.21.15.40:0/2999128994 shutdown_connections 2023-10-16T08:40:51.359 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.341+0000 7fa8cc43b640 1 -- 172.21.15.40:0/2999128994 >> 172.21.15.40:0/2999128994 conn(0x7fa8c40ce960 msgr2=0x7fa8c40ced70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:51.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.341+0000 7fa8cc43b640 1 -- 172.21.15.40:0/2999128994 shutdown_connections 2023-10-16T08:40:51.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.341+0000 7fa8cc43b640 1 -- 172.21.15.40:0/2999128994 wait complete. 2023-10-16T08:40:51.360 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8cc43b640 1 Processor -- start 2023-10-16T08:40:51.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8cc43b640 1 -- start start 2023-10-16T08:40:51.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8cc43b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b00039f0 0x7fa8c4151f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:51.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8cc43b640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8c4156a50 0x7fa8c4156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:51.361 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8cc43b640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8c40ac8e0 0x7fa8c40acd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:51.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8cc43b640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c40ad2a0 con 0x7fa8c40ac8e0 2023-10-16T08:40:51.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8cc43b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c4153260 con 0x7fa8b00039f0 2023-10-16T08:40:51.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8cc43b640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8c4157510 con 0x7fa8c4156a50 2023-10-16T08:40:51.362 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8ca9d8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8c40ac8e0 0x7fa8c40acd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:51.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8ca9d8640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8c40ac8e0 0x7fa8c40acd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39914/0 (socket says 172.21.15.40:39914) 2023-10-16T08:40:51.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8ca9d8640 1 -- 172.21.15.40:0/3326494925 learned_addr learned my addr 172.21.15.40:0/3326494925 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:40:51.363 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8c99d6640 1 --2- 172.21.15.40:0/3326494925 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8c4156a50 0x7fa8c4156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:51.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8ca1d7640 1 --2- 172.21.15.40:0/3326494925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b00039f0 0x7fa8c4151f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:51.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8ca9d8640 1 -- 172.21.15.40:0/3326494925 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8c4156a50 msgr2=0x7fa8c4156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:51.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8ca9d8640 1 --2- 172.21.15.40:0/3326494925 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8c4156a50 0x7fa8c4156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.364 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8ca9d8640 1 -- 172.21.15.40:0/3326494925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b00039f0 msgr2=0x7fa8c4151f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:51.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8ca9d8640 1 --2- 172.21.15.40:0/3326494925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b00039f0 0x7fa8c4151f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8ca9d8640 1 -- 172.21.15.40:0/3326494925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8c41577f0 con 0x7fa8c40ac8e0 2023-10-16T08:40:51.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8ca9d8640 1 --2- 172.21.15.40:0/3326494925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8c40ac8e0 0x7fa8c40acd60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc000c10 tx=0x7fa8bc00e820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:51.365 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8baffd640 1 -- 172.21.15.40:0/3326494925 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc001b90 con 0x7fa8c40ac8e0 2023-10-16T08:40:51.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8baffd640 1 -- 172.21.15.40:0/3326494925 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7fa8bc010510 con 0x7fa8c40ac8e0 2023-10-16T08:40:51.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8cc43b640 1 -- 172.21.15.40:0/3326494925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8c4157ad0 con 0x7fa8c40ac8e0 2023-10-16T08:40:51.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8baffd640 1 -- 172.21.15.40:0/3326494925 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8bc010900 con 0x7fa8c40ac8e0 2023-10-16T08:40:51.366 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8cc43b640 1 -- 172.21.15.40:0/3326494925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8c4152460 con 0x7fa8c40ac8e0 2023-10-16T08:40:51.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8cc43b640 1 -- 172.21.15.40:0/3326494925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8c404e9c0 con 0x7fa8c40ac8e0 2023-10-16T08:40:51.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.345+0000 7fa8baffd640 1 -- 172.21.15.40:0/3326494925 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7fa8bc010a60 con 0x7fa8c40ac8e0 2023-10-16T08:40:51.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.349+0000 7fa8baffd640 1 --2- 172.21.15.40:0/3326494925 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa8a80664d0 0x7fa8a8068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:51.367 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.349+0000 7fa8baffd640 1 -- 172.21.15.40:0/3326494925 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5854+0+0 (secure 0 0 0) 0x7fa8bc083300 con 0x7fa8c40ac8e0 2023-10-16T08:40:51.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.349+0000 7fa8ca1d7640 1 --2- 172.21.15.40:0/3326494925 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa8a80664d0 0x7fa8a8068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:51.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.349+0000 7fa8ca1d7640 1 --2- 172.21.15.40:0/3326494925 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa8a80664d0 0x7fa8a8068990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00af000 tx=0x7fa8c00a7040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:51.368 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.353+0000 7fa8baffd640 1 -- 172.21.15.40:0/3326494925 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7fa8bc050660 con 0x7fa8c40ac8e0 2023-10-16T08:40:51.546 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.541+0000 7fa8cc43b640 1 -- 172.21.15.40:0/3326494925 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa8c415eb90 con 0x7fa8c40ac8e0 2023-10-16T08:40:51.549 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.545+0000 7fa8baffd640 1 -- 172.21.15.40:0/3326494925 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v7) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa8bc019020 con 0x7fa8c40ac8e0 2023-10-16T08:40:51.550 INFO:teuthology.orchestra.run.smithi040.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-16T08:40:51.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.549+0000 7fa8cc43b640 1 -- 172.21.15.40:0/3326494925 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa8a80664d0 msgr2=0x7fa8a8068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:51.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.549+0000 7fa8cc43b640 1 --2- 172.21.15.40:0/3326494925 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa8a80664d0 0x7fa8a8068990 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00af000 tx=0x7fa8c00a7040 comp rx=0 tx=0).stop 2023-10-16T08:40:51.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.549+0000 7fa8cc43b640 1 -- 172.21.15.40:0/3326494925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8c40ac8e0 msgr2=0x7fa8c40acd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:51.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.549+0000 7fa8cc43b640 1 --2- 172.21.15.40:0/3326494925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8c40ac8e0 0x7fa8c40acd60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fa8bc000c10 tx=0x7fa8bc00e820 comp rx=0 tx=0).stop 2023-10-16T08:40:51.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.549+0000 7fa8cc43b640 1 -- 172.21.15.40:0/3326494925 shutdown_connections 2023-10-16T08:40:51.554 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.549+0000 7fa8cc43b640 1 --2- 172.21.15.40:0/3326494925 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7fa8a80664d0 0x7fa8a8068990 secure :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa8c00af000 tx=0x7fa8c00a7040 comp rx=0 tx=0).stop 2023-10-16T08:40:51.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.549+0000 7fa8cc43b640 1 --2- 172.21.15.40:0/3326494925 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7fa8c40ac8e0 0x7fa8c40acd60 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.549+0000 7fa8cc43b640 1 --2- 172.21.15.40:0/3326494925 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7fa8c4156a50 0x7fa8c4156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.549+0000 7fa8cc43b640 1 --2- 172.21.15.40:0/3326494925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa8b00039f0 0x7fa8c4151f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.555 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.549+0000 7fa8cc43b640 1 -- 172.21.15.40:0/3326494925 >> 172.21.15.40:0/3326494925 conn(0x7fa8c40ce960 msgr2=0x7fa8c4154e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:51.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.549+0000 7fa8cc43b640 1 -- 172.21.15.40:0/3326494925 shutdown_connections 2023-10-16T08:40:51.556 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.549+0000 7fa8cc43b640 1 -- 172.21.15.40:0/3326494925 wait complete. 2023-10-16T08:40:51.563 INFO:tasks.ceph.mds.c.smithi040.stderr: -4950> 2023-10-16T08:38:49.572+0000 7f5d8abb6640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-16T08:40:51.571 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-16T08:40:51.605 INFO:tasks.ceph.mds.c.smithi040.stdout:starting mds.c at 2023-10-16T08:40:51.706 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.701+0000 7f0635607640 1 Processor -- start 2023-10-16T08:40:51.707 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.701+0000 7f0635607640 1 -- start start 2023-10-16T08:40:51.708 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f0635607640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0630159dd0 0x7f06300c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:51.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f0635607640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063015b490 con 0x7f06300adec0 2023-10-16T08:40:51.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f0635607640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f063006d230 con 0x7f06300c9f00 2023-10-16T08:40:51.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f0635607640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f063006cd70 con 0x7f0630159dd0 2023-10-16T08:40:51.709 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f062effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0630159dd0 0x7f06300c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:51.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f062effd640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f0630159dd0 0x7f06300c8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39934/0 (socket says 172.21.15.40:39934) 2023-10-16T08:40:51.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f062e7fc640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f06300adec0 0x7f06300c97f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46306/0 (socket says 172.21.15.40:46306) 2023-10-16T08:40:51.710 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f062effd640 1 -- 172.21.15.40:0/4132457916 learned_addr learned my addr 172.21.15.40:0/4132457916 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:40:51.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f062effd640 1 -- 172.21.15.40:0/4132457916 >> v1:172.21.15.86:6789/0 conn(0x7f06300c9f00 legacy=0x7f06300cc460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:51.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f062effd640 1 -- 172.21.15.40:0/4132457916 >> v1:172.21.15.40:6789/0 conn(0x7f06300adec0 legacy=0x7f06300c97f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:51.711 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f062effd640 1 -- 172.21.15.40:0/4132457916 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f063006cf50 con 0x7f0630159dd0 2023-10-16T08:40:51.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f062effd640 1 --2- 172.21.15.40:0/4132457916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0630159dd0 0x7f06300c8f70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0620009870 tx=0x7f062001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42e2604a2cb6fe98 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:51.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f062dffb640 1 -- 172.21.15.40:0/4132457916 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06200ab600 con 0x7f0630159dd0 2023-10-16T08:40:51.713 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f062dffb640 1 -- 172.21.15.40:0/4132457916 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f06200a7050 con 0x7f0630159dd0 2023-10-16T08:40:51.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f062dffb640 1 -- 172.21.15.40:0/4132457916 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06200ab9a0 con 0x7f0630159dd0 2023-10-16T08:40:51.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f0635607640 1 -- 172.21.15.40:0/4132457916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0630159dd0 msgr2=0x7f06300c8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:51.714 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f0635607640 1 --2- 172.21.15.40:0/4132457916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0630159dd0 0x7f06300c8f70 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f0620009870 tx=0x7f062001c540 comp rx=0 tx=0).stop 2023-10-16T08:40:51.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f0635607640 1 -- 172.21.15.40:0/4132457916 shutdown_connections 2023-10-16T08:40:51.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f0635607640 1 --2- 172.21.15.40:0/4132457916 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0630159dd0 0x7f06300c8f70 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.715 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f0635607640 1 -- 172.21.15.40:0/4132457916 >> 172.21.15.40:0/4132457916 conn(0x7f06300ce890 msgr2=0x7f06300ceca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:51.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f0635607640 1 -- 172.21.15.40:0/4132457916 shutdown_connections 2023-10-16T08:40:51.716 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.705+0000 7f0635607640 1 -- 172.21.15.40:0/4132457916 wait complete. 2023-10-16T08:40:51.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f0635607640 1 Processor -- start 2023-10-16T08:40:51.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f0635607640 1 -- start start 2023-10-16T08:40:51.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f0635607640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f06300adec0 0x7f063015d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:51.717 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f0635607640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06300c9f00 0x7f06301514b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:51.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f0635607640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0630159dd0 0x7f0630151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:51.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f0635607640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0630158940 con 0x7f0630159dd0 2023-10-16T08:40:51.718 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f0635607640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f063015d140 con 0x7f06300c9f00 2023-10-16T08:40:51.719 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f0635607640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f063015c530 con 0x7f06300adec0 2023-10-16T08:40:51.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f062f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0630159dd0 0x7f0630151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:51.721 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f062f7fe640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0630159dd0 0x7f0630151d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39938/0 (socket says 172.21.15.40:39938) 2023-10-16T08:40:51.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f062f7fe640 1 -- 172.21.15.40:0/2654640271 learned_addr learned my addr 172.21.15.40:0/2654640271 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:40:51.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f062effd640 1 --2- 172.21.15.40:0/2654640271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f06300adec0 0x7f063015d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:51.722 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f062e7fc640 1 --2- 172.21.15.40:0/2654640271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06300c9f00 0x7f06301514b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f062f7fe640 1 -- 172.21.15.40:0/2654640271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f06300adec0 msgr2=0x7f063015d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f062f7fe640 1 --2- 172.21.15.40:0/2654640271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f06300adec0 0x7f063015d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f062f7fe640 1 -- 172.21.15.40:0/2654640271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06300c9f00 msgr2=0x7f06301514b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:51.723 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f062f7fe640 1 --2- 172.21.15.40:0/2654640271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06300c9f00 0x7f06301514b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:51.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f062f7fe640 1 -- 172.21.15.40:0/2654640271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06200094d0 con 0x7f0630159dd0 2023-10-16T08:40:51.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f062f7fe640 1 --2- 172.21.15.40:0/2654640271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0630159dd0 0x7f0630151d30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f062800c730 tx=0x7f062800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:51.724 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f06277fe640 1 -- 172.21.15.40:0/2654640271 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0628003b80 con 0x7f0630159dd0 2023-10-16T08:40:51.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f06277fe640 1 -- 172.21.15.40:0/2654640271 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f0628003ce0 con 0x7f0630159dd0 2023-10-16T08:40:51.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f0635607640 1 -- 172.21.15.40:0/2654640271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f063015c810 con 0x7f0630159dd0 2023-10-16T08:40:51.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f06277fe640 1 -- 172.21.15.40:0/2654640271 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0628005330 con 0x7f0630159dd0 2023-10-16T08:40:51.725 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.709+0000 7f0635607640 1 -- 172.21.15.40:0/2654640271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f063015cdd0 con 0x7f0630159dd0 2023-10-16T08:40:51.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.713+0000 7f0635607640 1 -- 172.21.15.40:0/2654640271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0630067a20 con 0x7f0630159dd0 2023-10-16T08:40:51.726 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.713+0000 7f06277fe640 1 -- 172.21.15.40:0/2654640271 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f06280056a0 con 0x7f0630159dd0 2023-10-16T08:40:51.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.713+0000 7f06277fe640 1 --2- 172.21.15.40:0/2654640271 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f06100664a0 0x7f0610068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:51.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.713+0000 7f06277fe640 1 -- 172.21.15.40:0/2654640271 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f0628083e80 con 0x7f0630159dd0 2023-10-16T08:40:51.727 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.713+0000 7f062effd640 1 --2- 172.21.15.40:0/2654640271 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f06100664a0 0x7f0610068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.713+0000 7f062effd640 1 --2- 172.21.15.40:0/2654640271 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f06100664a0 0x7f0610068960 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0630071d10 tx=0x7f0620008040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:51.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.717+0000 7f06277fe640 1 -- 172.21.15.40:0/2654640271 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f0628051230 con 0x7f0630159dd0 2023-10-16T08:40:51.917 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:51.913+0000 7f0635607640 1 -- 172.21.15.40:0/2654640271 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f06300ce690 con 0x7f0630159dd0 2023-10-16T08:40:52.553 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.549+0000 7f06277fe640 1 -- 172.21.15.40:0/2654640271 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v8) v1 ==== 104+0+0 (secure 0 0 0) 0x7f0628054850 con 0x7f0630159dd0 2023-10-16T08:40:52.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.553+0000 7f0635607640 1 -- 172.21.15.40:0/2654640271 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f06100664a0 msgr2=0x7f0610068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:52.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.553+0000 7f0635607640 1 --2- 172.21.15.40:0/2654640271 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f06100664a0 0x7f0610068960 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0630071d10 tx=0x7f0620008040 comp rx=0 tx=0).stop 2023-10-16T08:40:52.557 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.553+0000 7f0635607640 1 -- 172.21.15.40:0/2654640271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0630159dd0 msgr2=0x7f0630151d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:52.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.553+0000 7f0635607640 1 --2- 172.21.15.40:0/2654640271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0630159dd0 0x7f0630151d30 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f062800c730 tx=0x7f062800cbf0 comp rx=0 tx=0).stop 2023-10-16T08:40:52.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.553+0000 7f0635607640 1 -- 172.21.15.40:0/2654640271 shutdown_connections 2023-10-16T08:40:52.558 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.553+0000 7f0635607640 1 --2- 172.21.15.40:0/2654640271 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f06100664a0 0x7f0610068960 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.553+0000 7f0635607640 1 --2- 172.21.15.40:0/2654640271 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f0630159dd0 0x7f0630151d30 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.553+0000 7f0635607640 1 --2- 172.21.15.40:0/2654640271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06300c9f00 0x7f06301514b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.553+0000 7f0635607640 1 --2- 172.21.15.40:0/2654640271 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f06300adec0 0x7f063015d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:52.559 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.553+0000 7f0635607640 1 -- 172.21.15.40:0/2654640271 >> 172.21.15.40:0/2654640271 conn(0x7f06300ce890 msgr2=0x7f0630070060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.553+0000 7f0635607640 1 -- 172.21.15.40:0/2654640271 shutdown_connections 2023-10-16T08:40:52.560 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.553+0000 7f0635607640 1 -- 172.21.15.40:0/2654640271 wait complete. 2023-10-16T08:40:52.581 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-16T08:40:52.765 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.761+0000 7f1b325e5640 1 Processor -- start 2023-10-16T08:40:52.766 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.761+0000 7f1b325e5640 1 -- start start 2023-10-16T08:40:52.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.761+0000 7f1b325e5640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1b2c0d1400 0x7f1b2c0b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:52.767 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.761+0000 7f1b325e5640 1 -- --> v1:172.21.15.40:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b2c15d3c0 con 0x7f1b2c15cfe0 2023-10-16T08:40:52.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.761+0000 7f1b325e5640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b2c0d19b0 con 0x7f1b2c0b4cb0 2023-10-16T08:40:52.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.761+0000 7f1b325e5640 1 -- --> v2:172.21.15.40:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b2c0d1c90 con 0x7f1b2c0d1400 2023-10-16T08:40:52.768 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.761+0000 7f1b30b82640 1 --2- >> v2:172.21.15.40:3300/0 conn(0x7f1b2c0d1400 0x7f1b2c0b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:52.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.761+0000 7f1b2bfff640 1 --1- >> v1:172.21.15.40:6789/0 conn(0x7f1b2c15cfe0 0x7f1b2c0b45a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.40:6789/0 says I am v1:172.21.15.40:46318/0 (socket says 172.21.15.40:46318) 2023-10-16T08:40:52.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.761+0000 7f1b2bfff640 1 -- 172.21.15.40:0/603256752 learned_addr learned my addr 172.21.15.40:0/603256752 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:40:52.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.765+0000 7f1b2affd640 1 -- 172.21.15.40:0/603256752 <== mon.0 v1:172.21.15.40:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2015073805 0 0) 0x7f1b2c15d3c0 con 0x7f1b2c15cfe0 2023-10-16T08:40:52.769 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.765+0000 7f1b2affd640 1 -- 172.21.15.40:0/603256752 --> v1:172.21.15.40:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b14003530 con 0x7f1b2c15cfe0 2023-10-16T08:40:52.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.765+0000 7f1b30b82640 1 -- 172.21.15.40:0/603256752 >> v1:172.21.15.86:6789/0 conn(0x7f1b2c0b4cb0 legacy=0x7f1b2c0d0cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:40:52.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.765+0000 7f1b30b82640 1 -- 172.21.15.40:0/603256752 >> v1:172.21.15.40:6789/0 conn(0x7f1b2c15cfe0 legacy=0x7f1b2c0b45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:52.770 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.765+0000 7f1b30b82640 1 -- 172.21.15.40:0/603256752 --> v2:172.21.15.40:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2c0d1f70 con 0x7f1b2c0d1400 2023-10-16T08:40:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.765+0000 7f1b30b82640 1 --2- 172.21.15.40:0/603256752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b2c0d1400 0x7f1b2c0b6d40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009d90 tx=0x7f1b2401c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb20b3ddd3ac7a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.765+0000 7f1b2affd640 1 -- 172.21.15.40:0/603256752 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b240a58c0 con 0x7f1b2c0d1400 2023-10-16T08:40:52.772 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.765+0000 7f1b2affd640 1 -- 172.21.15.40:0/603256752 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1b240a5a20 con 0x7f1b2c0d1400 2023-10-16T08:40:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.765+0000 7f1b2affd640 1 -- 172.21.15.40:0/603256752 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b240a5dd0 con 0x7f1b2c0d1400 2023-10-16T08:40:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.765+0000 7f1b325e5640 1 -- 172.21.15.40:0/603256752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b2c0d1400 msgr2=0x7f1b2c0b6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:52.773 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.765+0000 7f1b325e5640 1 --2- 172.21.15.40:0/603256752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b2c0d1400 0x7f1b2c0b6d40 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f1b24009d90 tx=0x7f1b2401c910 comp rx=0 tx=0).stop 2023-10-16T08:40:52.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.765+0000 7f1b325e5640 1 -- 172.21.15.40:0/603256752 shutdown_connections 2023-10-16T08:40:52.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.765+0000 7f1b325e5640 1 --2- 172.21.15.40:0/603256752 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b2c0d1400 0x7f1b2c0b6d40 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:52.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.765+0000 7f1b325e5640 1 -- 172.21.15.40:0/603256752 >> 172.21.15.40:0/603256752 conn(0x7f1b2c0cf5a0 msgr2=0x7f1b2c0cf9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:52.774 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.765+0000 7f1b325e5640 1 -- 172.21.15.40:0/603256752 shutdown_connections 2023-10-16T08:40:52.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.765+0000 7f1b325e5640 1 -- 172.21.15.40:0/603256752 wait complete. 2023-10-16T08:40:52.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b325e5640 1 Processor -- start 2023-10-16T08:40:52.775 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b325e5640 1 -- start start 2023-10-16T08:40:52.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b325e5640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b2c0b4cb0 0x7f1b2c147e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:52.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b325e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b2c0d1400 0x7f1b2c1f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:52.778 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b325e5640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b2c15cfe0 0x7f1b2c1f8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:52.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b325e5640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c152540 con 0x7f1b2c15cfe0 2023-10-16T08:40:52.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b325e5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c152820 con 0x7f1b2c0d1400 2023-10-16T08:40:52.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b325e5640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2c152b00 con 0x7f1b2c0b4cb0 2023-10-16T08:40:52.779 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b30b82640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b2c15cfe0 0x7f1b2c1f8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:52.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b30b82640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b2c15cfe0 0x7f1b2c1f8bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39960/0 (socket says 172.21.15.40:39960) 2023-10-16T08:40:52.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b30b82640 1 -- 172.21.15.40:0/1467295362 learned_addr learned my addr 172.21.15.40:0/1467295362 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:40:52.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b2b7fe640 1 --2- 172.21.15.40:0/1467295362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b2c0d1400 0x7f1b2c1f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:52.780 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b2bfff640 1 --2- 172.21.15.40:0/1467295362 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b2c0b4cb0 0x7f1b2c147e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:52.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b30b82640 1 -- 172.21.15.40:0/1467295362 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b2c0b4cb0 msgr2=0x7f1b2c147e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:52.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b30b82640 1 --2- 172.21.15.40:0/1467295362 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b2c0b4cb0 0x7f1b2c147e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:52.781 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b30b82640 1 -- 172.21.15.40:0/1467295362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b2c0d1400 msgr2=0x7f1b2c1f8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:52.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b30b82640 1 --2- 172.21.15.40:0/1467295362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b2c0d1400 0x7f1b2c1f8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:52.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b30b82640 1 -- 172.21.15.40:0/1467295362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b24009a70 con 0x7f1b2c15cfe0 2023-10-16T08:40:52.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b30b82640 1 --2- 172.21.15.40:0/1467295362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b2c15cfe0 0x7f1b2c1f8bc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1b240b0040 tx=0x7f1b240196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:52.782 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b28ff9640 1 -- 172.21.15.40:0/1467295362 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b240ad850 con 0x7f1b2c15cfe0 2023-10-16T08:40:52.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b28ff9640 1 -- 172.21.15.40:0/1467295362 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f1b240ad9b0 con 0x7f1b2c15cfe0 2023-10-16T08:40:52.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b28ff9640 1 -- 172.21.15.40:0/1467295362 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b240add80 con 0x7f1b2c15cfe0 2023-10-16T08:40:52.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b325e5640 1 -- 172.21.15.40:0/1467295362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b2c152de0 con 0x7f1b2c15cfe0 2023-10-16T08:40:52.783 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b325e5640 1 -- 172.21.15.40:0/1467295362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b2c1533a0 con 0x7f1b2c15cfe0 2023-10-16T08:40:52.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.769+0000 7f1b325e5640 1 -- 172.21.15.40:0/1467295362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b2c153590 con 0x7f1b2c15cfe0 2023-10-16T08:40:52.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.773+0000 7f1b28ff9640 1 -- 172.21.15.40:0/1467295362 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7f1b240b4050 con 0x7f1b2c15cfe0 2023-10-16T08:40:52.784 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.773+0000 7f1b28ff9640 1 --2- 172.21.15.40:0/1467295362 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1b180664a0 0x7f1b18068960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:52.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.773+0000 7f1b28ff9640 1 -- 172.21.15.40:0/1467295362 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5940+0+0 (secure 0 0 0) 0x7f1b2411f930 con 0x7f1b2c15cfe0 2023-10-16T08:40:52.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.773+0000 7f1b2bfff640 1 --2- 172.21.15.40:0/1467295362 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1b180664a0 0x7f1b18068960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:52.785 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.773+0000 7f1b2bfff640 1 --2- 172.21.15.40:0/1467295362 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1b180664a0 0x7f1b18068960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c005890 tx=0x7f1b1c009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:52.786 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.777+0000 7f1b28ff9640 1 -- 172.21.15.40:0/1467295362 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7f1b240ecc10 con 0x7f1b2c15cfe0 2023-10-16T08:40:52.938 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:52.933+0000 7f1b325e5640 1 -- 172.21.15.40:0/1467295362 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1b2c153590 con 0x7f1b2c15cfe0 2023-10-16T08:40:53.564 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.561+0000 7f1b28ff9640 1 -- 172.21.15.40:0/1467295362 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v24) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1b2411de60 con 0x7f1b2c15cfe0 2023-10-16T08:40:53.564 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_metadata' removed 2023-10-16T08:40:53.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.561+0000 7f1b325e5640 1 -- 172.21.15.40:0/1467295362 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1b180664a0 msgr2=0x7f1b18068960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:53.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.561+0000 7f1b325e5640 1 --2- 172.21.15.40:0/1467295362 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1b180664a0 0x7f1b18068960 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f1b1c005890 tx=0x7f1b1c009040 comp rx=0 tx=0).stop 2023-10-16T08:40:53.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.561+0000 7f1b325e5640 1 -- 172.21.15.40:0/1467295362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b2c15cfe0 msgr2=0x7f1b2c1f8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:53.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.561+0000 7f1b325e5640 1 --2- 172.21.15.40:0/1467295362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b2c15cfe0 0x7f1b2c1f8bc0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f1b240b0040 tx=0x7f1b240196b0 comp rx=0 tx=0).stop 2023-10-16T08:40:53.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.561+0000 7f1b325e5640 1 -- 172.21.15.40:0/1467295362 shutdown_connections 2023-10-16T08:40:53.568 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.561+0000 7f1b325e5640 1 --2- 172.21.15.40:0/1467295362 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7f1b180664a0 0x7f1b18068960 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:53.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.561+0000 7f1b325e5640 1 --2- 172.21.15.40:0/1467295362 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7f1b2c15cfe0 0x7f1b2c1f8bc0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:53.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.561+0000 7f1b325e5640 1 --2- 172.21.15.40:0/1467295362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b2c0d1400 0x7f1b2c1f8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:53.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.561+0000 7f1b325e5640 1 --2- 172.21.15.40:0/1467295362 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7f1b2c0b4cb0 0x7f1b2c147e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:53.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.561+0000 7f1b325e5640 1 -- 172.21.15.40:0/1467295362 >> 172.21.15.40:0/1467295362 conn(0x7f1b2c0cf5a0 msgr2=0x7f1b2c0b61b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:53.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.565+0000 7f1b325e5640 1 -- 172.21.15.40:0/1467295362 shutdown_connections 2023-10-16T08:40:53.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.565+0000 7f1b325e5640 1 -- 172.21.15.40:0/1467295362 wait complete. 2023-10-16T08:40:53.598 DEBUG:teuthology.orchestra.run.smithi040:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-16T08:40:53.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.725+0000 7ff47eec4640 1 Processor -- start 2023-10-16T08:40:53.728 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.725+0000 7ff47eec4640 1 -- start start 2023-10-16T08:40:53.729 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.725+0000 7ff47eec4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff4780ad820 0x7ff47806f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:53.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.725+0000 7ff47eec4640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff4780add90 0x7ff4780cd040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:53.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.725+0000 7ff47eec4640 1 -- --> v1:172.21.15.86:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff47806b7d0 con 0x7ff4780c9940 2023-10-16T08:40:53.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.725+0000 7ff47eec4640 1 -- --> v2:172.21.15.86:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff47806bab0 con 0x7ff4780add90 2023-10-16T08:40:53.730 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.725+0000 7ff47eec4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff47806fe30 con 0x7ff4780ad820 2023-10-16T08:40:53.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.725+0000 7ff47d461640 1 --2- >> v2:172.21.15.86:3300/0 conn(0x7ff4780add90 0x7ff4780cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:53.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.725+0000 7ff477fff640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff4780ad820 0x7ff47806f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:53.731 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.725+0000 7ff47cc60640 1 --1- >> v1:172.21.15.86:6789/0 conn(0x7ff4780c9940 0x7ff47815cde0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.86:6789/0 says I am v1:172.21.15.40:35872/0 (socket says 172.21.15.40:35872) 2023-10-16T08:40:53.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.725+0000 7ff47cc60640 1 -- 172.21.15.40:0/3969225202 learned_addr learned my addr 172.21.15.40:0/3969225202 (peer_addr_for_me v1:172.21.15.40:0/0) 2023-10-16T08:40:53.732 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.725+0000 7ff477fff640 1 -- 172.21.15.40:0/3969225202 >> v2:172.21.15.86:3300/0 conn(0x7ff4780add90 msgr2=0x7ff4780cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-16T08:40:53.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.725+0000 7ff477fff640 1 --2- 172.21.15.40:0/3969225202 >> v2:172.21.15.86:3300/0 conn(0x7ff4780add90 0x7ff4780cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:53.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.725+0000 7ff477fff640 1 -- 172.21.15.40:0/3969225202 >> v1:172.21.15.86:6789/0 conn(0x7ff4780c9940 legacy=0x7ff47815cde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:53.733 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.725+0000 7ff477fff640 1 -- 172.21.15.40:0/3969225202 --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4780c90a0 con 0x7ff4780ad820 2023-10-16T08:40:53.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.729+0000 7ff477fff640 1 --2- 172.21.15.40:0/3969225202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff4780ad820 0x7ff47806f8f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0097c0 tx=0x7ff46c01c420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a08f4c7f3d93cb98 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:53.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.729+0000 7ff4777fe640 1 -- 172.21.15.40:0/3969225202 <== mon.2 v2:172.21.15.86:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c0ab600 con 0x7ff4780ad820 2023-10-16T08:40:53.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.729+0000 7ff4777fe640 1 -- 172.21.15.40:0/3969225202 <== mon.2 v2:172.21.15.86:3301/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff46c0a7050 con 0x7ff4780ad820 2023-10-16T08:40:53.734 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.729+0000 7ff4777fe640 1 -- 172.21.15.40:0/3969225202 <== mon.2 v2:172.21.15.86:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff46c0ab9a0 con 0x7ff4780ad820 2023-10-16T08:40:53.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.729+0000 7ff47eec4640 1 -- 172.21.15.40:0/3969225202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff4780ad820 msgr2=0x7ff47806f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:53.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.729+0000 7ff47eec4640 1 --2- 172.21.15.40:0/3969225202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff4780ad820 0x7ff47806f8f0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff46c0097c0 tx=0x7ff46c01c420 comp rx=0 tx=0).stop 2023-10-16T08:40:53.737 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.729+0000 7ff47eec4640 1 -- 172.21.15.40:0/3969225202 shutdown_connections 2023-10-16T08:40:53.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.729+0000 7ff47eec4640 1 --2- 172.21.15.40:0/3969225202 >> v2:172.21.15.86:3300/0 conn(0x7ff4780add90 0x7ff4780cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:53.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.729+0000 7ff47eec4640 1 --2- 172.21.15.40:0/3969225202 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff4780ad820 0x7ff47806f8f0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:53.738 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.729+0000 7ff47eec4640 1 -- 172.21.15.40:0/3969225202 >> 172.21.15.40:0/3969225202 conn(0x7ff4780ce8b0 msgr2=0x7ff4780cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:53.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.729+0000 7ff47eec4640 1 -- 172.21.15.40:0/3969225202 shutdown_connections 2023-10-16T08:40:53.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.729+0000 7ff47eec4640 1 -- 172.21.15.40:0/3969225202 wait complete. 2023-10-16T08:40:53.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.729+0000 7ff47eec4640 1 Processor -- start 2023-10-16T08:40:53.739 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.729+0000 7ff47eec4640 1 -- start start 2023-10-16T08:40:53.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff47eec4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4780ad820 0x7ff4781586f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:53.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff47eec4640 1 --2- >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff4780add90 0x7ff478156a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:53.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff47eec4640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4780c9940 0x7ff4781512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:53.740 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff47eec4640 1 -- --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4780c9480 con 0x7ff4780c9940 2023-10-16T08:40:53.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff47eec4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff478158c30 con 0x7ff4780ad820 2023-10-16T08:40:53.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff47eec4640 1 -- --> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff478156f50 con 0x7ff4780add90 2023-10-16T08:40:53.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff47d461640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4780c9940 0x7ff4781512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:53.741 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff47d461640 1 --2- >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4780c9940 0x7ff4781512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.40:3300/0 says I am v2:172.21.15.40:39970/0 (socket says 172.21.15.40:39970) 2023-10-16T08:40:53.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff47d461640 1 -- 172.21.15.40:0/3338460859 learned_addr learned my addr 172.21.15.40:0/3338460859 (peer_addr_for_me v2:172.21.15.40:0/0) 2023-10-16T08:40:53.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff477fff640 1 --2- 172.21.15.40:0/3338460859 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff4780add90 0x7ff478156a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:53.742 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff47cc60640 1 --2- 172.21.15.40:0/3338460859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4780ad820 0x7ff4781586f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:53.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff47d461640 1 -- 172.21.15.40:0/3338460859 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff4780add90 msgr2=0x7ff478156a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:53.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff47d461640 1 --2- 172.21.15.40:0/3338460859 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff4780add90 0x7ff478156a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:53.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff47d461640 1 -- 172.21.15.40:0/3338460859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4780ad820 msgr2=0x7ff4781586f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:53.743 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff47d461640 1 --2- 172.21.15.40:0/3338460859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4780ad820 0x7ff4781586f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:53.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff47d461640 1 -- 172.21.15.40:0/3338460859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff46c009420 con 0x7ff4780c9940 2023-10-16T08:40:53.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff47d461640 1 --2- 172.21.15.40:0/3338460859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4780c9940 0x7ff4781512b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff468009dd0 tx=0x7ff46800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:53.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff4757fa640 1 -- 172.21.15.40:0/3338460859 <== mon.0 v2:172.21.15.40:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468014480 con 0x7ff4780c9940 2023-10-16T08:40:53.744 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff4757fa640 1 -- 172.21.15.40:0/3338460859 <== mon.0 v2:172.21.15.40:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7ff46800f040 con 0x7ff4780c9940 2023-10-16T08:40:53.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff47eec4640 1 -- 172.21.15.40:0/3338460859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff478157a10 con 0x7ff4780c9940 2023-10-16T08:40:53.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff4757fa640 1 -- 172.21.15.40:0/3338460859 <== mon.0 v2:172.21.15.40:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff468014860 con 0x7ff4780c9940 2023-10-16T08:40:53.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff47eec4640 1 -- 172.21.15.40:0/3338460859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff478151850 con 0x7ff4780c9940 2023-10-16T08:40:53.745 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff47eec4640 1 -- 172.21.15.40:0/3338460859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff478067a20 con 0x7ff4780c9940 2023-10-16T08:40:53.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff4757fa640 1 -- 172.21.15.40:0/3338460859 <== mon.0 v2:172.21.15.40:3300/0 4 ==== mgrmap(e 5) v1 ==== 85850+0+0 (secure 0 0 0) 0x7ff4680149c0 con 0x7ff4780c9940 2023-10-16T08:40:53.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff4757fa640 1 --2- 172.21.15.40:0/3338460859 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff4640664d0 0x7ff464068990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-16T08:40:53.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff4757fa640 1 -- 172.21.15.40:0/3338460859 <== mon.0 v2:172.21.15.40:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5502+0+0 (secure 0 0 0) 0x7ff468084ab0 con 0x7ff4780c9940 2023-10-16T08:40:53.747 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.733+0000 7ff47cc60640 1 --2- 172.21.15.40:0/3338460859 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff4640664d0 0x7ff464068990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-16T08:40:53.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.737+0000 7ff47cc60640 1 --2- 172.21.15.40:0/3338460859 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff4640664d0 0x7ff464068990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff4700058e0 tx=0x7ff470009040 comp rx=0 tx=0).ready entity=mgr.4111 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-16T08:40:53.748 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.737+0000 7ff4757fa640 1 -- 172.21.15.40:0/3338460859 <== mon.0 v2:172.21.15.40:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172989 (secure 0 0 0) 0x7ff468051e50 con 0x7ff4780c9940 2023-10-16T08:40:53.905 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:53.901+0000 7ff47eec4640 1 -- 172.21.15.40:0/3338460859 --> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff47804e9c0 con 0x7ff4780c9940 2023-10-16T08:40:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:54.561+0000 7ff4757fa640 1 -- 172.21.15.40:0/3338460859 <== mon.0 v2:172.21.15.40:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v25) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff4680180d0 con 0x7ff4780c9940 2023-10-16T08:40:54.567 INFO:teuthology.orchestra.run.smithi040.stderr:pool 'cephfs_data' removed 2023-10-16T08:40:54.569 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:54.565+0000 7ff47eec4640 1 -- 172.21.15.40:0/3338460859 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff4640664d0 msgr2=0x7ff464068990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:54.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:54.565+0000 7ff47eec4640 1 --2- 172.21.15.40:0/3338460859 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff4640664d0 0x7ff464068990 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7ff4700058e0 tx=0x7ff470009040 comp rx=0 tx=0).stop 2023-10-16T08:40:54.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:54.565+0000 7ff47eec4640 1 -- 172.21.15.40:0/3338460859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4780c9940 msgr2=0x7ff4781512b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-16T08:40:54.570 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:54.565+0000 7ff47eec4640 1 --2- 172.21.15.40:0/3338460859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4780c9940 0x7ff4781512b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7ff468009dd0 tx=0x7ff46800c490 comp rx=0 tx=0).stop 2023-10-16T08:40:54.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:54.565+0000 7ff47eec4640 1 -- 172.21.15.40:0/3338460859 shutdown_connections 2023-10-16T08:40:54.571 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:54.565+0000 7ff47eec4640 1 --2- 172.21.15.40:0/3338460859 >> [v2:172.21.15.86:6832/1188970899,v1:172.21.15.86:6833/1188970899] conn(0x7ff4640664d0 0x7ff464068990 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:54.565+0000 7ff47eec4640 1 --2- 172.21.15.40:0/3338460859 >> [v2:172.21.15.40:3300/0,v1:172.21.15.40:6789/0] conn(0x7ff4780c9940 0x7ff4781512b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:54.565+0000 7ff47eec4640 1 --2- 172.21.15.40:0/3338460859 >> [v2:172.21.15.86:3301/0,v1:172.21.15.86:6790/0] conn(0x7ff4780add90 0x7ff478156a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:54.572 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:54.565+0000 7ff47eec4640 1 --2- 172.21.15.40:0/3338460859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff4780ad820 0x7ff4781586f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-16T08:40:54.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:54.565+0000 7ff47eec4640 1 -- 172.21.15.40:0/3338460859 >> 172.21.15.40:0/3338460859 conn(0x7ff4780ce8b0 msgr2=0x7ff4780cc460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-16T08:40:54.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:54.565+0000 7ff47eec4640 1 -- 172.21.15.40:0/3338460859 shutdown_connections 2023-10-16T08:40:54.573 INFO:teuthology.orchestra.run.smithi040.stderr:2023-10-16T08:40:54.565+0000 7ff47eec4640 1 -- 172.21.15.40:0/3338460859 wait complete. 2023-10-16T08:40:54.602 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-16T08:40:54.603 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-16T08:40:54.603 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:40:54.615 INFO:tasks.ceph.mds.a:Stopped 2023-10-16T08:40:54.616 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-16T08:40:54.616 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:40:54.675 INFO:tasks.ceph.mds.c:Stopped 2023-10-16T08:40:54.675 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-16T08:40:54.676 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:40:54.687 INFO:tasks.ceph.mds.b:Stopped 2023-10-16T08:40:54.688 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-16T08:40:54.688 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:40:54.750 INFO:tasks.ceph.mds.d:Stopped 2023-10-16T08:40:54.750 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-16T08:40:54.751 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-16T08:40:54.751 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:40:54.828 INFO:tasks.ceph.osd.0:Stopped 2023-10-16T08:40:54.829 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-16T08:40:54.829 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:40:54.912 INFO:tasks.ceph.osd.1:Stopped 2023-10-16T08:40:54.913 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-16T08:40:54.913 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:40:55.002 INFO:tasks.ceph.osd.2:Stopped 2023-10-16T08:40:55.003 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-16T08:40:55.003 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:40:57.911 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 is failed for ~0s 2023-10-16T08:41:01.105 INFO:tasks.ceph.osd.3:Stopped 2023-10-16T08:41:01.106 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-16T08:41:01.106 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:41:01.177 INFO:tasks.ceph.osd.4:Stopped 2023-10-16T08:41:01.177 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-16T08:41:01.178 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:41:01.255 INFO:tasks.ceph.osd.5:Stopped 2023-10-16T08:41:01.255 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-16T08:41:01.256 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:41:01.330 INFO:tasks.ceph.osd.6:Stopped 2023-10-16T08:41:01.330 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-16T08:41:01.331 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:41:01.412 INFO:tasks.ceph.osd.7:Stopped 2023-10-16T08:41:01.413 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-16T08:41:01.413 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-16T08:41:01.414 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:41:01.441 INFO:tasks.ceph.mgr.y:Stopped 2023-10-16T08:41:01.442 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-16T08:41:01.442 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:41:01.471 INFO:tasks.ceph.mgr.x:Stopped 2023-10-16T08:41:01.472 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-16T08:41:01.472 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-16T08:41:01.472 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:41:01.499 INFO:tasks.ceph.mon.a:Stopped 2023-10-16T08:41:01.499 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-16T08:41:01.500 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:41:01.524 INFO:tasks.ceph.mon.b:Stopped 2023-10-16T08:41:01.525 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-16T08:41:01.525 INFO:teuthology.orchestra.run:waiting for 300 2023-10-16T08:41:01.587 INFO:tasks.ceph.mon.c:Stopped 2023-10-16T08:41:01.588 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-16T08:41:01.588 DEBUG:teuthology.orchestra.run.smithi040:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-10-16T08:41:01.606 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi040.front.sepia.ceph.com 2023-10-16T08:41:01.606 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-16T08:41:01.889 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi040.front.sepia.ceph.com 2023-10-16T08:41:01.889 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-16T08:41:02.062 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi040.front.sepia.ceph.com 2023-10-16T08:41:02.063 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-16T08:41:02.197 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi040.front.sepia.ceph.com 2023-10-16T08:41:02.197 DEBUG:teuthology.orchestra.run.smithi040:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-16T08:41:02.330 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi086.front.sepia.ceph.com 2023-10-16T08:41:02.330 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-16T08:41:02.621 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi086.front.sepia.ceph.com 2023-10-16T08:41:02.622 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-16T08:41:02.817 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi086.front.sepia.ceph.com 2023-10-16T08:41:02.818 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-16T08:41:02.913 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.3 has been restored 2023-10-16T08:41:02.992 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi086.front.sepia.ceph.com 2023-10-16T08:41:02.992 DEBUG:teuthology.orchestra.run.smithi086:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-16T08:41:03.174 INFO:tasks.ceph:Archiving mon data... 2023-10-16T08:41:03.178 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/mchangir-2023-10-16_08:10:12-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428951/data/mon.a.tgz 2023-10-16T08:41:03.179 DEBUG:teuthology.orchestra.run.smithi040:> mktemp 2023-10-16T08:41:03.186 INFO:teuthology.orchestra.run.smithi040.stdout:/tmp/tmp.46YLT9tyfi 2023-10-16T08:41:03.186 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.46YLT9tyfi 2023-10-16T08:41:03.293 DEBUG:teuthology.orchestra.run.smithi040:> sudo chmod 0666 /tmp/tmp.46YLT9tyfi 2023-10-16T08:41:03.356 DEBUG:teuthology.orchestra.remote:smithi040:/tmp/tmp.46YLT9tyfi is 266KB 2023-10-16T08:41:03.418 DEBUG:teuthology.orchestra.run.smithi040:> rm -fr /tmp/tmp.46YLT9tyfi 2023-10-16T08:41:03.423 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/mchangir-2023-10-16_08:10:12-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428951/data/mon.b.tgz 2023-10-16T08:41:03.424 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2023-10-16T08:41:03.430 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.KsW3lI2KOr 2023-10-16T08:41:03.431 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.KsW3lI2KOr 2023-10-16T08:41:03.555 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.KsW3lI2KOr 2023-10-16T08:41:03.616 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.KsW3lI2KOr is 360KB 2023-10-16T08:41:03.678 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.KsW3lI2KOr 2023-10-16T08:41:03.684 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/mchangir-2023-10-16_08:10:12-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428951/data/mon.c.tgz 2023-10-16T08:41:03.685 DEBUG:teuthology.orchestra.run.smithi086:> mktemp 2023-10-16T08:41:03.733 INFO:teuthology.orchestra.run.smithi086.stdout:/tmp/tmp.tDq7twchnO 2023-10-16T08:41:03.733 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.tDq7twchnO 2023-10-16T08:41:03.858 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0666 /tmp/tmp.tDq7twchnO 2023-10-16T08:41:03.925 DEBUG:teuthology.orchestra.remote:smithi086:/tmp/tmp.tDq7twchnO is 360KB 2023-10-16T08:41:03.943 DEBUG:teuthology.orchestra.run.smithi086:> rm -fr /tmp/tmp.tDq7twchnO 2023-10-16T08:41:03.948 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-16T08:41:03.948 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-16T08:41:03.951 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-16T08:41:04.000 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-16T08:41:04.000 DEBUG:teuthology.orchestra.run.smithi040:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-16T08:41:04.009 DEBUG:teuthology.orchestra.run.smithi086:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-16T08:41:04.032 INFO:teuthology.orchestra.run.smithi040.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-16T08:41:04.067 INFO:teuthology.orchestra.run.smithi086.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-16T08:41:04.071 INFO:tasks.ceph:Archiving crash dumps... 2023-10-16T08:41:04.073 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2023-10-16_08:10:12-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428951/remote/smithi040/crash 2023-10-16T08:41:04.074 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-16T08:41:04.092 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/crash to /home/teuthworker/archive/mchangir-2023-10-16_08:10:12-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428951/remote/smithi086/crash 2023-10-16T08:41:04.093 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-16T08:41:04.122 INFO:tasks.ceph:Shutting down logrotate 2023-10-16T08:41:04.123 DEBUG:teuthology.orchestra.run.smithi040:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:41:04.137 DEBUG:teuthology.orchestra.run.smithi086:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-16T08:41:04.206 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-16T08:41:04.231 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-16T08:41:04.264 INFO:tasks.ceph:Compressing logs... 2023-10-16T08:41:04.265 DEBUG:teuthology.orchestra.run.smithi040:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-16T08:41:04.279 DEBUG:teuthology.orchestra.run.smithi086:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2023-10-16T08:41:04.302 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44506.log 2023-10-16T08:41:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46745.log 2023-10-16T08:41:04.304 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44506.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38136.log 2023-10-16T08:41:04.305 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46745.log: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.44506.log.gz 2023-10-16T08:41:04.305 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43904.log 2023-10-16T08:41:04.305 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46745.log.gz 2023-10-16T08:41:04.306 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.38136.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43502.log 2023-10-16T08:41:04.310 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.38136.log.gz 2023-10-16T08:41:04.310 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43904.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44097.log 2023-10-16T08:41:04.310 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43502.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.43904.log.gz 2023-10-16T08:41:04.311 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46381.log 2023-10-16T08:41:04.311 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43502.log.gz 2023-10-16T08:41:04.311 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44097.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44688.log 2023-10-16T08:41:04.311 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44097.log.gz 2023-10-16T08:41:04.312 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44792.log 2023-10-16T08:41:04.312 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46381.log: /var/log/ceph/ceph-client.admin.44688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43901.log 2023-10-16T08:41:04.312 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44558.log 2023-10-16T08:41:04.312 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44688.log.gz 2023-10-16T08:41:04.313 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.46381.log.gz 2023-10-16T08:41:04.313 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43901.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.44792.log.gz 2023-10-16T08:41:04.313 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2023-10-16T08:41:04.313 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43901.log.gz 2023-10-16T08:41:04.314 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47547.log 2023-10-16T08:41:04.314 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8%/var/log/ceph/ceph-mgr.y.log: -- replaced with /var/log/ceph/ceph-client.admin.44558.log.gz 2023-10-16T08:41:04.314 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47076.log 2023-10-16T08:41:04.315 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47547.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46011.log 2023-10-16T08:41:04.315 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47547.log.gz 2023-10-16T08:41:04.315 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47076.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43700.log 2023-10-16T08:41:04.315 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44532.log 2023-10-16T08:41:04.316 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.47076.log.gz 2023-10-16T08:41:04.316 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43700.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46011.log.gz 2023-10-16T08:41:04.316 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43344.log 2023-10-16T08:41:04.316 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43700.log.gz 2023-10-16T08:41:04.332 INFO:teuthology.orchestra.run.smithi040.stderr:gzip/var/log/ceph/ceph-client.admin.44532.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.46511.log 2023-10-16T08:41:04.332 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45795.log 2023-10-16T08:41:04.333 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44532.log.gz 2023-10-16T08:41:04.333 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46511.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43344.log.gz 2023-10-16T08:41:04.333 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45978.log 2023-10-16T08:41:04.334 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46511.log.gz 2023-10-16T08:41:04.334 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2023-10-16T08:41:04.334 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45795.log: /var/log/ceph/ceph-client.admin.45978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45715.log 2023-10-16T08:41:04.334 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45795.log.gz 2023-10-16T08:41:04.335 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45978.log.gz 2023-10-16T08:41:04.335 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46433.log 2023-10-16T08:41:04.335 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43726.log 2023-10-16T08:41:04.335 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45715.log.gz 2023-10-16T08:41:04.336 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46433.log: 93.0% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2023-10-16T08:41:04.336 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47258.log 2023-10-16T08:41:04.336 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46433.log.gz 2023-10-16T08:41:04.337 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44326.log 2023-10-16T08:41:04.337 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43726.log: /var/log/ceph/ceph-client.admin.47258.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44114.log 2023-10-16T08:41:04.337 INFO:teuthology.orchestra.run.smithi040.stderr: 87.7% -- replaced with /var/log/ceph/ceph-client.admin.43726.log.gz 2023-10-16T08:41:04.337 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47258.log.gz 2023-10-16T08:41:04.338 INFO:teuthology.orchestra.run.smithi040.stderr:gzip/var/log/ceph/ceph-client.admin.44326.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.43909.log 2023-10-16T08:41:04.338 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44114.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46615.log 2023-10-16T08:41:04.338 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44326.log.gz 2023-10-16T08:41:04.338 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.44114.log.gz 2023-10-16T08:41:04.339 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43778.log 2023-10-16T08:41:04.339 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.43909.log.gz 2023-10-16T08:41:04.339 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46998.log 2023-10-16T08:41:04.340 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46615.log: /var/log/ceph/ceph-client.admin.43778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43900.log 2023-10-16T08:41:04.340 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46615.log.gz 2023-10-16T08:41:04.340 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46998.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43778.log.gz 2023-10-16T08:41:04.340 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43280.log 2023-10-16T08:41:04.341 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46998.log.gz 2023-10-16T08:41:04.341 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45900.log 2023-10-16T08:41:04.341 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43900.log: /var/log/ceph/ceph-client.admin.43280.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46168.log 2023-10-16T08:41:04.342 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45900.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43900.log.gz 2023-10-16T08:41:04.342 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43398.log 2023-10-16T08:41:04.342 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.43280.log.gz 2023-10-16T08:41:04.342 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45900.log.gz 2023-10-16T08:41:04.343 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46168.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47024.log 2023-10-16T08:41:04.343 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46168.log.gz 2023-10-16T08:41:04.343 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45077.log 2023-10-16T08:41:04.343 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43398.log: /var/log/ceph/ceph-client.admin.47024.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45531.log 2023-10-16T08:41:04.344 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.0.45077.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.43398.log.gz 2023-10-16T08:41:04.344 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2023-10-16T08:41:04.344 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47024.log.gz 2023-10-16T08:41:04.345 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45531.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44323.log 2023-10-16T08:41:04.345 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.45531.log.gz 2023-10-16T08:41:04.345 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.2.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.45077.log.gz 2023-10-16T08:41:04.345 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46589.log 2023-10-16T08:41:04.346 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44740.log 2023-10-16T08:41:04.346 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44323.log: /var/log/ceph/ceph-client.admin.46589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46563.log 2023-10-16T08:41:04.346 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44323.log.gz 2023-10-16T08:41:04.346 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46589.log.gz 2023-10-16T08:41:04.347 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46840.log 2023-10-16T08:41:04.347 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.44740.log.gz 2023-10-16T08:41:04.347 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47362.log 2023-10-16T08:41:04.348 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46563.log: /var/log/ceph/ceph-client.admin.46840.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46089.log 2023-10-16T08:41:04.348 INFO:teuthology.orchestra.run.smithi040.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.46840.log.gz 2023-10-16T08:41:04.348 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.46563.log.gz 2023-10-16T08:41:04.348 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45663.log 2023-10-16T08:41:04.349 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47154.log 2023-10-16T08:41:04.349 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47362.log.gz 2023-10-16T08:41:04.349 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46089.log: /var/log/ceph/ceph-client.admin.45663.log: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46089.log.gz 2023-10-16T08:41:04.349 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2023-10-16T08:41:04.350 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.45663.log.gz 2023-10-16T08:41:04.350 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46920.log 2023-10-16T08:41:04.350 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47154.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47573.log 2023-10-16T08:41:04.350 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47154.log.gz 2023-10-16T08:41:04.351 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46920.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47443.log 2023-10-16T08:41:04.351 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.46920.log.gz 2023-10-16T08:41:04.351 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45479.log 2023-10-16T08:41:04.352 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45505.log 2023-10-16T08:41:04.352 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47443.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.47573.log.gz 2023-10-16T08:41:04.352 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45479.log: gzip -5 88.3% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.47443.log.gz 2023-10-16T08:41:04.352 INFO:teuthology.orchestra.run.smithi040.stderr: /var/log/ceph/ceph-client.admin.44714.log 2023-10-16T08:41:04.353 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45479.log.gz 2023-10-16T08:41:04.353 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45505.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46220.log 2023-10-16T08:41:04.353 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.45505.log.gz 2023-10-16T08:41:04.353 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45349.log 2023-10-16T08:41:04.354 INFO:teuthology.orchestra.run.smithi040.stderr:gzip/var/log/ceph/ceph-client.admin.46220.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.44636.log 2023-10-16T08:41:04.354 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.44714.log.gz 2023-10-16T08:41:04.354 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45349.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.46220.log.gz 2023-10-16T08:41:04.354 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43646.log 2023-10-16T08:41:04.355 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.45349.log.gz 2023-10-16T08:41:04.355 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44636.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44818.log 2023-10-16T08:41:04.355 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43646.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44636.log.gz 2023-10-16T08:41:04.356 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46641.log 2023-10-16T08:41:04.356 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43646.log.gz 2023-10-16T08:41:04.356 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44818.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47284.log 2023-10-16T08:41:04.356 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46641.log: 88.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46485.log 2023-10-16T08:41:04.357 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44818.log.gz 87.7% 2023-10-16T08:41:04.357 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46641.log.gz 2023-10-16T08:41:04.357 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47284.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43830.log 2023-10-16T08:41:04.357 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46485.log: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.47284.log.gz 2023-10-16T08:41:04.358 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45926.log 2023-10-16T08:41:04.358 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.46485.log.gz 2023-10-16T08:41:04.358 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43476.log 2023-10-16T08:41:04.359 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43830.log: /var/log/ceph/ceph-client.admin.45926.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43752.log 2023-10-16T08:41:04.359 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.43830.log.gz 2023-10-16T08:41:04.359 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45926.log.gz 2023-10-16T08:41:04.359 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45291.log 2023-10-16T08:41:04.360 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43476.log.gz 2023-10-16T08:41:04.360 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43752.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43424.log 2023-10-16T08:41:04.374 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38226.log 2023-10-16T08:41:04.374 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2023-10-16T08:41:04.374 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38226.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2023-10-16T08:41:04.375 INFO:teuthology.orchestra.run.smithi086.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.38226.log.gz 2023-10-16T08:41:04.375 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.1.44060.log 2023-10-16T08:41:04.375 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38202.log 2023-10-16T08:41:04.376 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.1.44060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38274.log 2023-10-16T08:41:04.376 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38202.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2023-10-16T08:41:04.376 INFO:teuthology.orchestra.run.smithi086.stderr: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38202.log.gz 2023-10-16T08:41:04.376 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38274.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38371.log 2023-10-16T08:41:04.377 INFO:teuthology.orchestra.run.smithi086.stderr: 90.8% -- replaced with /var/log/ceph/ceph-client.1.44060.log.gz 2023-10-16T08:41:04.377 INFO:teuthology.orchestra.run.smithi086.stderr: 87.6% -- replaced with /var/log/ceph/ceph-client.admin.38274.log.gz 2023-10-16T08:41:04.377 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose --/var/log/ceph/ceph-osd.5.log: /var/log/ceph/ceph-client.admin.38323.log 2023-10-16T08:41:04.378 INFO:teuthology.orchestra.run.smithi086.stderr:gzip/var/log/ceph/ceph-client.admin.38371.log: -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2023-10-16T08:41:04.378 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38323.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.38371.log.gz 2023-10-16T08:41:04.378 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-16T08:41:04.378 INFO:teuthology.orchestra.run.smithi086.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.38323.log.gz 2023-10-16T08:41:04.379 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.3.44175.log 2023-10-16T08:41:04.379 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2023-10-16T08:41:04.379 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.3.44175.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38250.log 2023-10-16T08:41:04.379 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38299.log 2023-10-16T08:41:04.380 INFO:teuthology.orchestra.run.smithi086.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.3.44175.log.gz 2023-10-16T08:41:04.380 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38250.log: 91.8% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-16T08:41:04.380 INFO:teuthology.orchestra.run.smithi086.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.38250.log.gz 2023-10-16T08:41:04.381 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-16T08:41:04.381 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.38299.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.d.log 2023-10-16T08:41:04.381 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph.audit.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.38299.log.gz 2023-10-16T08:41:04.381 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mds.b.log 2023-10-16T08:41:04.382 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mds.d.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2023-10-16T08:41:04.382 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mds.b.log: 91.8% -- replaced with /var/log/ceph/ceph-mds.d.log.gz 2023-10-16T08:41:04.383 INFO:teuthology.orchestra.run.smithi086.stderr: 93.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-16T08:41:04.383 INFO:teuthology.orchestra.run.smithi086.stderr: 91.9%gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2023-10-16T08:41:04.384 INFO:teuthology.orchestra.run.smithi086.stderr: -- replaced with /var/log/ceph/ceph-mds.b.log.gz 2023-10-16T08:41:04.384 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-mon.b.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38347.log 2023-10-16T08:41:04.384 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-osd.4.log: /var/log/ceph/ceph-client.admin.38347.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.38347.log.gz 2023-10-16T08:41:04.393 INFO:teuthology.orchestra.run.smithi086.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2023-10-16T08:41:04.414 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45124.log 2023-10-16T08:41:04.415 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45291.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43752.log.gz 2023-10-16T08:41:04.415 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43424.log: gzip 87.9% -- replaced with /var/log/ceph/ceph-client.admin.45291.log.gz -5 2023-10-16T08:41:04.415 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-osd.3.log 2023-10-16T08:41:04.416 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.43424.log.gz 2023-10-16T08:41:04.416 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45124.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46063.log 2023-10-16T08:41:04.416 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.45124.log.gz -5 2023-10-16T08:41:04.417 INFO:teuthology.orchestra.run.smithi040.stderr: --verbose -- /var/log/ceph/ceph-client.admin.43910.log 2023-10-16T08:41:04.417 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.46063.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47495.log 2023-10-16T08:41:04.417 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3%/var/log/ceph/ceph-client.admin.43910.log: -- replaced with /var/log/ceph/ceph-client.admin.46063.log.gz 2023-10-16T08:41:04.418 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43899.log 2023-10-16T08:41:04.418 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.43910.log.gz 2023-10-16T08:41:04.418 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44301.log 2023-10-16T08:41:04.419 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47495.log.gz/var/log/ceph/ceph-client.admin.43899.log: 2023-10-16T08:41:04.419 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43560.log 2023-10-16T08:41:04.419 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44301.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43899.log.gz 2023-10-16T08:41:04.420 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46459.log 2023-10-16T08:41:04.420 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43560.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44301.log.gz 2023-10-16T08:41:04.420 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45323.log 2023-10-16T08:41:04.421 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46459.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43560.log.gz 2023-10-16T08:41:04.421 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44317.log 2023-10-16T08:41:04.421 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45323.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46719.log 2023-10-16T08:41:04.421 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46459.log.gz 2023-10-16T08:41:04.422 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45323.log.gz 2023-10-16T08:41:04.422 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44317.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46972.log 2023-10-16T08:41:04.422 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7%gzip -5/var/log/ceph/ceph-client.admin.46719.log: --verbose -- /var/log/ceph/ceph-client.admin.44896.log 2023-10-16T08:41:04.423 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44317.log.gz 2023-10-16T08:41:04.423 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46972.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46719.log.gz 2023-10-16T08:41:04.423 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44319.log 2023-10-16T08:41:04.424 INFO:teuthology.orchestra.run.smithi040.stderr: 87.6%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46693.log 2023-10-16T08:41:04.424 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.46972.log.gz/var/log/ceph/ceph-client.admin.44896.log: 2023-10-16T08:41:04.424 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44319.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2023-10-16T08:41:04.425 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45427.log 2023-10-16T08:41:04.425 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.44319.log.gz 2023-10-16T08:41:04.425 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46693.log: 90.9% -- replaced with /var/log/ceph/ceph-client.admin.44896.log.gz 2023-10-16T08:41:04.425 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-osd.1.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46693.log.gz 2023-10-16T08:41:04.426 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43889.log 2023-10-16T08:41:04.426 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46115.log 2023-10-16T08:41:04.427 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.45427.log.gz/var/log/ceph/ceph-client.admin.43889.log: 2023-10-16T08:41:04.427 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44115.log 2023-10-16T08:41:04.427 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45265.log 2023-10-16T08:41:04.428 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.43889.log.gz 2023-10-16T08:41:04.428 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46115.log: /var/log/ceph/ceph-client.admin.44115.log: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.46115.log.gz 2023-10-16T08:41:04.428 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.44115.log.gz 2023-10-16T08:41:04.428 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46142.log 2023-10-16T08:41:04.429 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45265.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47336.log 2023-10-16T08:41:04.429 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7%/var/log/ceph/ceph-client.admin.46142.log: -- replaced with /var/log/ceph/ceph-client.admin.45265.log.gz 2023-10-16T08:41:04.429 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44324.log 2023-10-16T08:41:04.430 INFO:teuthology.orchestra.run.smithi040.stderr: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46142.log.gz 2023-10-16T08:41:04.430 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47336.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47336.log.gz 2023-10-16T08:41:04.431 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47310.log 2023-10-16T08:41:04.431 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44324.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46303.log 2023-10-16T08:41:04.431 INFO:teuthology.orchestra.run.smithi040.stderr: 87.5%/var/log/ceph/ceph-client.admin.47310.log: -- replaced with /var/log/ceph/ceph-client.admin.44324.log.gz 2023-10-16T08:41:04.431 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43856.log 2023-10-16T08:41:04.432 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.47310.log.gz 2023-10-16T08:41:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46303.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46303.log.gz 2023-10-16T08:41:04.432 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46407.log 2023-10-16T08:41:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43856.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47390.log 2023-10-16T08:41:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46407.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.43856.log.gz 2023-10-16T08:41:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45821.log 2023-10-16T08:41:04.433 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43532.log 2023-10-16T08:41:04.434 INFO:teuthology.orchestra.run.smithi040.stderr: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.46407.log.gz 2023-10-16T08:41:04.434 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47390.log: /var/log/ceph/ceph-client.admin.45821.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.47390.log.gz 2023-10-16T08:41:04.435 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.45821.log.gz 2023-10-16T08:41:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46277.log 2023-10-16T08:41:04.435 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43532.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45375.log 2023-10-16T08:41:04.436 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2%/var/log/ceph/ceph-client.admin.46277.log: -- replaced with /var/log/ceph/ceph-client.admin.43532.log.gzgzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.33163.log 2023-10-16T08:41:04.436 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-10-16T08:41:04.436 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46277.log.gz 2023-10-16T08:41:04.437 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45375.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.45375.log.gz 2023-10-16T08:41:04.437 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47644.log 2023-10-16T08:41:04.437 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph.tmp-client.admin.33163.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2023-10-16T08:41:04.437 INFO:teuthology.orchestra.run.smithi040.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.33163.log.gz 2023-10-16T08:41:04.438 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45637.log 2023-10-16T08:41:04.438 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2%/var/log/ceph/ceph.log: -- replaced with /var/log/ceph/ceph-client.admin.47644.log.gz 2023-10-16T08:41:04.438 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43804.log 2023-10-16T08:41:04.439 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45637.log: 89.4% -- replaced with /var/log/ceph/ceph-client.admin.45637.log.gz 2023-10-16T08:41:04.439 INFO:teuthology.orchestra.run.smithi040.stderr: 85.6% -- replaced with /var/log/ceph/ceph.log.gz 2023-10-16T08:41:04.439 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47206.log 2023-10-16T08:41:04.439 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43804.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47419.log 2023-10-16T08:41:04.440 INFO:teuthology.orchestra.run.smithi040.stderr: 89.6%/var/log/ceph/ceph-client.admin.47206.log: -- replaced with /var/log/ceph/ceph-client.admin.43804.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43903.log 2023-10-16T08:41:04.440 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-10-16T08:41:04.440 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47206.log.gz 2023-10-16T08:41:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47419.log: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.47419.log.gz 2023-10-16T08:41:04.441 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44870.log 2023-10-16T08:41:04.442 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46246.log 2023-10-16T08:41:04.442 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44870.log: 89.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45743.log 2023-10-16T08:41:04.443 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.43903.log.gz 2023-10-16T08:41:04.443 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.44870.log.gz 2023-10-16T08:41:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46246.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46246.log.gz 2023-10-16T08:41:04.443 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47232.log 2023-10-16T08:41:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45743.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44118.log 2023-10-16T08:41:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47232.log: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45743.log.gz 2023-10-16T08:41:04.444 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.2.45215.log 2023-10-16T08:41:04.445 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44118.log: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.47232.log.gz 2023-10-16T08:41:04.445 INFO:teuthology.orchestra.run.smithi040.stderr: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.44118.log.gz 2023-10-16T08:41:04.446 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45952.log 2023-10-16T08:41:04.446 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.2.45215.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45585.log 2023-10-16T08:41:04.446 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45952.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43674.log 2023-10-16T08:41:04.447 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.45952.log.gz/var/log/ceph/ceph-client.admin.45585.log: 2023-10-16T08:41:04.447 INFO:teuthology.orchestra.run.smithi040.stderr: 90.7% -- replaced with /var/log/ceph/ceph-client.2.45215.log.gz 2023-10-16T08:41:04.447 INFO:teuthology.orchestra.run.smithi040.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.45585.log.gz 2023-10-16T08:41:04.448 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44662.log 2023-10-16T08:41:04.448 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43674.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38250.log 2023-10-16T08:41:04.448 INFO:teuthology.orchestra.run.smithi040.stderr: 87.4%/var/log/ceph/ceph-client.admin.44662.log: -- replaced with /var/log/ceph/ceph-client.admin.43674.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45689.log 2023-10-16T08:41:04.449 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-10-16T08:41:04.449 INFO:teuthology.orchestra.run.smithi040.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.44662.log.gz 2023-10-16T08:41:04.449 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.38250.log: 87.1% -- replaced with /var/log/ceph/ceph-client.admin.38250.log.gz 2023-10-16T08:41:04.450 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43450.log 2023-10-16T08:41:04.450 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45689.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43594.log 2023-10-16T08:41:04.450 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43450.log: 88.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43190.log 2023-10-16T08:41:04.450 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45689.log.gz 2023-10-16T08:41:04.451 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.43450.log.gz 2023-10-16T08:41:04.451 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43594.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.43594.log.gz 2023-10-16T08:41:04.451 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46771.log 2023-10-16T08:41:04.452 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43190.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45559.log 2023-10-16T08:41:04.452 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9%/var/log/ceph/ceph-client.admin.46771.log: -- replaced with /var/log/ceph/ceph-client.admin.43190.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47128.log 2023-10-16T08:41:04.453 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-10-16T08:41:04.453 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.46771.log.gz 2023-10-16T08:41:04.454 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45559.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45559.log.gz 2023-10-16T08:41:04.455 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2023-10-16T08:41:04.455 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47128.log: gzip -5 --verbose -- /var/log/ceph/ceph-mds.c.log 2023-10-16T08:41:04.456 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph.audit.log: 87.7%gzip -5 --verbose -- /var/log/ceph/ceph-mds.a.log 2023-10-16T08:41:04.456 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47128.log.gz 2023-10-16T08:41:04.457 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mds.c.log: 89.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2023-10-16T08:41:04.457 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44302.log 2023-10-16T08:41:04.457 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44112.log 2023-10-16T08:41:04.458 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-mds.a.log: /var/log/ceph/ceph-client.admin.44302.log: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44302.log.gz 2023-10-16T08:41:04.458 INFO:teuthology.orchestra.run.smithi040.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mds.a.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45401.log 2023-10-16T08:41:04.459 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-10-16T08:41:04.459 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45611.log 2023-10-16T08:41:04.459 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.44112.log.gz 2023-10-16T08:41:04.460 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45401.log: 89.4%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46946.log 2023-10-16T08:41:04.460 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.45401.log.gz 2023-10-16T08:41:04.461 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45611.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45769.log 2023-10-16T08:41:04.461 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.45611.log.gz 2023-10-16T08:41:04.461 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45847.log 2023-10-16T08:41:04.462 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.46946.log.gz 2023-10-16T08:41:04.462 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45769.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44610.log 2023-10-16T08:41:04.462 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.45769.log.gz 2023-10-16T08:41:04.463 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45847.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44325.log 2023-10-16T08:41:04.463 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.45847.log.gz 2023-10-16T08:41:04.463 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44610.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38015.log 2023-10-16T08:41:04.464 INFO:teuthology.orchestra.run.smithi040.stderr: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.44610.log.gz 2023-10-16T08:41:04.464 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46667.log 2023-10-16T08:41:04.464 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44325.log: /var/log/ceph/ceph-client.admin.38015.log: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44325.log.gz 2023-10-16T08:41:04.464 INFO:teuthology.orchestra.run.smithi040.stderr: 90.0% -- replaced with /var/log/ceph/ceph-client.admin.38015.log.gz 2023-10-16T08:41:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47102.log 2023-10-16T08:41:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46355.log 2023-10-16T08:41:04.465 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46667.log: /var/log/ceph/ceph-client.admin.47102.log: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.46667.log.gz 2023-10-16T08:41:04.466 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.47102.log.gz 2023-10-16T08:41:04.466 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44109.log 2023-10-16T08:41:04.466 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45874.log 2023-10-16T08:41:04.466 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46355.log: /var/log/ceph/ceph-client.admin.44109.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46355.log.gz 2023-10-16T08:41:04.467 INFO:teuthology.orchestra.run.smithi040.stderr: 88.6% -- replaced with /var/log/ceph/ceph-client.admin.44109.log.gz 2023-10-16T08:41:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47521.log 2023-10-16T08:41:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45874.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.45874.log.gz 2023-10-16T08:41:04.467 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46194.log 2023-10-16T08:41:04.468 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47521.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46329.log 2023-10-16T08:41:04.468 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0%/var/log/ceph/ceph-client.admin.46194.log: -- replaced with /var/log/ceph/ceph-client.admin.47521.log.gz 2023-10-16T08:41:04.468 INFO:teuthology.orchestra.run.smithi040.stderr: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46194.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47618.log 2023-10-16T08:41:04.469 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-10-16T08:41:04.469 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45453.log 2023-10-16T08:41:04.469 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.46329.log.gz 2023-10-16T08:41:04.469 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47618.log: 87.5%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44951.log 2023-10-16T08:41:04.470 INFO:teuthology.orchestra.run.smithi040.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47618.log.gz 2023-10-16T08:41:04.470 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.45453.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47180.log 2023-10-16T08:41:04.470 INFO:teuthology.orchestra.run.smithi040.stderr: 88.5% -- replaced with /var/log/ceph/ceph-client.admin.45453.log.gz 2023-10-16T08:41:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46894.log 2023-10-16T08:41:04.471 INFO:teuthology.orchestra.run.smithi040.stderr: 89.2% -- replaced with /var/log/ceph/ceph-client.admin.44951.log.gz 2023-10-16T08:41:04.471 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47180.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46866.log 2023-10-16T08:41:04.472 INFO:teuthology.orchestra.run.smithi040.stderr: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.47180.log.gz 2023-10-16T08:41:04.472 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44117.log 2023-10-16T08:41:04.472 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46894.log: /var/log/ceph/ceph-client.admin.46866.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46894.log.gz 2023-10-16T08:41:04.472 INFO:teuthology.orchestra.run.smithi040.stderr: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.46866.log.gz 2023-10-16T08:41:04.473 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44584.log 2023-10-16T08:41:04.473 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46037.log 2023-10-16T08:41:04.473 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44117.log: /var/log/ceph/ceph-client.admin.44584.log: 88.4% -- replaced with /var/log/ceph/ceph-client.admin.44117.log.gz 2023-10-16T08:41:04.474 INFO:teuthology.orchestra.run.smithi040.stderr: 89.3% -- replaced with /var/log/ceph/ceph-client.admin.44584.log.gz 2023-10-16T08:41:04.474 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46537.log 2023-10-16T08:41:04.474 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46037.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.46037.log.gz 2023-10-16T08:41:04.474 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44766.log 2023-10-16T08:41:04.475 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43620.log 2023-10-16T08:41:04.475 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46537.log: /var/log/ceph/ceph-client.admin.44766.log: 88.9% -- replaced with /var/log/ceph/ceph-client.admin.46537.log.gz 2023-10-16T08:41:04.475 INFO:teuthology.orchestra.run.smithi040.stderr: 88.3% -- replaced with /var/log/ceph/ceph-client.admin.44766.log.gz 2023-10-16T08:41:04.475 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47469.log 2023-10-16T08:41:04.476 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46814.log 2023-10-16T08:41:04.476 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.43620.log: /var/log/ceph/ceph-client.admin.47469.log: 88.2% -- replaced with /var/log/ceph/ceph-client.admin.43620.log.gz 2023-10-16T08:41:04.476 INFO:teuthology.orchestra.run.smithi040.stderr: 88.0% -- replaced with /var/log/ceph/ceph-client.admin.47469.log.gz 2023-10-16T08:41:04.477 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44107.log 2023-10-16T08:41:04.477 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44844.log 2023-10-16T08:41:04.478 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.46814.log: /var/log/ceph/ceph-client.admin.44107.log: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.46814.log.gz 2023-10-16T08:41:04.478 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.44107.log.gz 2023-10-16T08:41:04.479 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43247.log 2023-10-16T08:41:04.479 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47050.log 2023-10-16T08:41:04.479 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.44844.log: /var/log/ceph/ceph-client.admin.43247.log: 89.1% -- replaced with /var/log/ceph/ceph-client.admin.44844.log.gz 2023-10-16T08:41:04.480 INFO:teuthology.orchestra.run.smithi040.stderr: 88.8% -- replaced with /var/log/ceph/ceph-client.admin.43247.log.gz 2023-10-16T08:41:04.480 INFO:teuthology.orchestra.run.smithi040.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43370.log 2023-10-16T08:41:04.481 INFO:teuthology.orchestra.run.smithi040.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mds.c.log.gz 2023-10-16T08:41:04.481 INFO:teuthology.orchestra.run.smithi040.stderr:/var/log/ceph/ceph-client.admin.47050.log: /var/log/ceph/ceph-client.admin.43370.log: 89.0% -- replaced with /var/log/ceph/ceph-client.admin.47050.log.gz 2023-10-16T08:41:04.482 INFO:teuthology.orchestra.run.smithi040.stderr: 88.1% -- replaced with /var/log/ceph/ceph-client.admin.43370.log.gz 2023-10-16T08:41:04.484 INFO:teuthology.orchestra.run.smithi040.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2023-10-16T08:41:04.484 INFO:teuthology.orchestra.run.smithi040.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2023-10-16T08:41:04.485 INFO:teuthology.orchestra.run.smithi040.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2023-10-16T08:41:04.485 INFO:teuthology.orchestra.run.smithi040.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2023-10-16T08:41:04.485 INFO:teuthology.orchestra.run.smithi086.stderr: 91.8% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2023-10-16T08:41:04.485 INFO:teuthology.orchestra.run.smithi086.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2023-10-16T08:41:04.486 INFO:teuthology.orchestra.run.smithi086.stderr: 93.3% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2023-10-16T08:41:04.486 INFO:teuthology.orchestra.run.smithi086.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2023-10-16T08:41:04.487 INFO:teuthology.orchestra.run.smithi086.stderr: 93.4% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2023-10-16T08:41:04.501 INFO:teuthology.orchestra.run.smithi086.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2023-10-16T08:41:04.503 INFO:teuthology.orchestra.run.smithi086.stderr: 2023-10-16T08:41:04.503 INFO:teuthology.orchestra.run.smithi086.stderr:real 0m0.182s 2023-10-16T08:41:04.504 INFO:teuthology.orchestra.run.smithi086.stderr:user 0m0.696s 2023-10-16T08:41:04.504 INFO:teuthology.orchestra.run.smithi086.stderr:sys 0m0.082s 2023-10-16T08:41:04.511 INFO:teuthology.orchestra.run.smithi040.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2023-10-16T08:41:04.512 INFO:teuthology.orchestra.run.smithi040.stderr: 2023-10-16T08:41:04.513 INFO:teuthology.orchestra.run.smithi040.stderr:real 0m0.232s 2023-10-16T08:41:04.513 INFO:teuthology.orchestra.run.smithi040.stderr:user 0m0.848s 2023-10-16T08:41:04.513 INFO:teuthology.orchestra.run.smithi040.stderr:sys 0m0.092s 2023-10-16T08:41:04.514 INFO:tasks.ceph:Archiving logs... 2023-10-16T08:41:04.515 DEBUG:teuthology.misc:Transferring archived files from smithi040:/var/log/ceph to /home/teuthworker/archive/mchangir-2023-10-16_08:10:12-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428951/remote/smithi040/log 2023-10-16T08:41:04.517 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-16T08:41:04.773 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/mchangir-2023-10-16_08:10:12-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428951/remote/smithi086/log 2023-10-16T08:41:04.774 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-16T08:41:04.874 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-16T08:41:04.899 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-16T08:41:04.899 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-16T08:41:04.902 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-16T08:41:04.964 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-16T08:41:04.980 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-16T08:41:04.981 DEBUG:teuthology.orchestra.run.smithi040:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T08:41:04.984 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-16T08:41:04.993 INFO:teuthology.orchestra.run.smithi040.stdout: remote refid st t when poll reach delay offset jitter 2023-10-16T08:41:04.994 INFO:teuthology.orchestra.run.smithi040.stdout:============================================================================== 2023-10-16T08:41:04.994 INFO:teuthology.orchestra.run.smithi040.stdout:*hv01.front.sepi 44.4.53.2 2 u 32 64 77 0.123 -1.416 0.798 2023-10-16T08:41:04.994 INFO:teuthology.orchestra.run.smithi040.stdout:+hv02.front.sepi 63.231.80.2 3 u 29 64 77 0.079 +0.872 0.824 2023-10-16T08:41:04.995 INFO:teuthology.orchestra.run.smithi040.stdout:+hv03.front.sepi 74.6.168.72 3 u 30 64 77 0.135 +0.481 0.826 2023-10-16T08:41:04.995 INFO:teuthology.orchestra.run.smithi040.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T08:41:04.997 INFO:teuthology.orchestra.run.smithi086.stdout: remote refid st t when poll reach delay offset jitter 2023-10-16T08:41:04.998 INFO:teuthology.orchestra.run.smithi086.stdout:============================================================================== 2023-10-16T08:41:04.998 INFO:teuthology.orchestra.run.smithi086.stdout:*hv01.front.sepi 44.4.53.2 2 u 38 64 77 0.109 -1.759 1.606 2023-10-16T08:41:04.998 INFO:teuthology.orchestra.run.smithi086.stdout:+hv02.front.sepi 63.231.80.2 3 u 37 64 77 0.106 +0.529 1.634 2023-10-16T08:41:04.999 INFO:teuthology.orchestra.run.smithi086.stdout:+hv03.front.sepi 74.6.168.72 3 u 33 64 77 0.089 +0.246 1.719 2023-10-16T08:41:04.999 INFO:teuthology.orchestra.run.smithi086.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-10-16T08:41:05.000 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-16T08:41:05.014 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-16T08:41:05.014 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-16T08:41:05.063 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-16T08:41:05.114 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-16T08:41:05.164 INFO:teuthology.task.internal:Duration was 834.694656 seconds 2023-10-16T08:41:05.165 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-16T08:41:05.214 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-16T08:41:05.215 DEBUG:teuthology.orchestra.run.smithi040:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-16T08:41:05.218 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-16T08:41:05.283 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-16T08:41:05.284 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi040.front.sepia.ceph.com 2023-10-16T08:41:05.284 DEBUG:teuthology.orchestra.run.smithi040:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-16T08:41:05.295 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2023-10-16T08:41:05.295 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-16T08:41:05.305 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-16T08:41:05.306 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-16T08:41:05.341 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-16T08:41:05.373 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-16T08:41:05.374 DEBUG:teuthology.orchestra.run.smithi040:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-16T08:41:05.405 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-16T08:41:05.470 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-16T08:41:05.479 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-16T08:41:05.480 DEBUG:teuthology.orchestra.run.smithi040:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-16T08:41:05.491 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-16T08:41:05.502 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-16T08:41:05.523 DEBUG:teuthology.orchestra.run.smithi040:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-16T08:41:05.537 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-16T08:41:05.546 INFO:teuthology.orchestra.run.smithi040.stdout:kernel.core_pattern = core 2023-10-16T08:41:05.554 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2023-10-16T08:41:05.566 DEBUG:teuthology.orchestra.run.smithi040:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-16T08:41:05.607 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:41:05.608 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-16T08:41:05.612 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-16T08:41:05.613 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-16T08:41:05.623 INFO:teuthology.task.internal:Transferring archived files... 2023-10-16T08:41:05.624 DEBUG:teuthology.misc:Transferring archived files from smithi040:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2023-10-16_08:10:12-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428951/remote/smithi040 2023-10-16T08:41:05.624 DEBUG:teuthology.orchestra.run.smithi040:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-16T08:41:05.670 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/mchangir-2023-10-16_08:10:12-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428951/remote/smithi086 2023-10-16T08:41:05.671 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-16T08:41:05.688 INFO:teuthology.task.internal:Removing archive directory... 2023-10-16T08:41:05.688 DEBUG:teuthology.orchestra.run.smithi040:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-16T08:41:05.709 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-16T08:41:05.732 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-16T08:41:05.743 INFO:teuthology.task.internal:Not uploading archives. 2023-10-16T08:41:05.744 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-16T08:41:05.778 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-16T08:41:05.778 DEBUG:teuthology.orchestra.run.smithi040:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-16T08:41:05.781 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-16T08:41:05.785 INFO:teuthology.orchestra.run.smithi040.stdout: 655470 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 16 08:41 /home/ubuntu/cephtest 2023-10-16T08:41:05.787 INFO:teuthology.orchestra.run.smithi086.stdout: 655429 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 16 08:41 /home/ubuntu/cephtest 2023-10-16T08:41:05.788 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-16T08:41:05.823 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-16T08:41:05.958 INFO:teuthology.nuke:Checking targets against current locks 2023-10-16T08:41:05.984 DEBUG:teuthology.nuke:shortname: smithi040 2023-10-16T08:41:05.984 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-16T08:41:06.011 DEBUG:teuthology.nuke:shortname: smithi086 2023-10-16T08:41:06.012 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-16T08:41:06.015 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi040.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-10-16_08:10:12-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428951', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 08:18:54.782129', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC9NQL/MGgwiIOjSeKTp24cQ97ZwOsJMk9LstleBGMYYh52MDJnTBeLIx4/B8MBdtdmz/qKA4NUnTfa70peh9nk='} 2023-10-16T08:41:06.038 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/mchangir-2023-10-16_08:10:12-fs-wip-mchangir-mgr-snap_schedule-support-subvol-and-group-arguments-distro-default-smithi/7428951', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-16 08:18:54.783752', 'locked_by': 'mchangir', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBIPx1U1nPl6AeNRotZqILZpt6k01E0TmhKhCzRMRxWKyrwm7FoM4l+t0BUkfIsIT2DsTx0DwhEW/OSe+eW4DzRo='} 2023-10-16T08:41:06.068 INFO:teuthology.orchestra.console.smithi040:Power off 2023-10-16T08:41:06.070 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-16T08:41:06.093 DEBUG:teuthology.orchestra.console.smithi040:power off output: Chassis Power Control: Down/Off 2023-10-16T08:41:06.094 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T08:41:06.110 INFO:teuthology.orchestra.console.smithi086:Power off 2023-10-16T08:41:06.111 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-16T08:41:06.121 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2023-10-16T08:41:06.133 DEBUG:teuthology.orchestra.console.smithi086:power off output: Chassis Power Control: Down/Off 2023-10-16T08:41:06.134 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T08:41:06.158 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2023-10-16T08:41:10.123 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T08:41:10.159 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T08:41:10.233 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is on 2023-10-16T08:41:10.271 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2023-10-16T08:41:14.236 DEBUG:teuthology.orchestra.console.smithi040:pexpect command: ipmitool -H smithi040.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T08:41:14.272 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T08:41:14.347 DEBUG:teuthology.orchestra.console.smithi040:check power output: Chassis Power is off 2023-10-16T08:41:14.384 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2023-10-16T08:41:14.449 INFO:teuthology.orchestra.console.smithi040:Power off completed 2023-10-16T08:41:18.386 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-16T08:41:18.496 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is off 2023-10-16T08:41:18.597 INFO:teuthology.orchestra.console.smithi086:Power off completed 2023-10-16T08:41:18.702 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} duration: 834.6946563720703 failure_reason: positional argument follows keyword argument (test_snap_schedules.py, line 586) flavor: default owner: mchangir sentry_event: https://sentry.ceph.com/organizations/ceph/?query=97de75c98ad146b3b8c10d9eaa391c68 status: fail success: false 2023-10-16T08:41:18.703 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-16T08:41:18.786 INFO:teuthology.run:FAIL